2016-05-17 21:29:56,214 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-17 21:29:56,247 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.detail.EveryEveryPipeAssemblyPlatformTest, with platform: hadoop2-tez
2016-05-17 21:29:56,778 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-17 21:29:57,107 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-17 21:29:57,377 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides:
2016-05-17 21:29:57,534 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:57,536 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:57,625 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: BB2A5AC6191E43C885E57B7553598D9B
2016-05-17 21:29:57,838 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.081
2016-05-17 21:29:57,851 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.095
2016-05-17 21:29:57,853 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:29:57,853 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:29:57,854 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:57,984 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:57,987 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1
2016-05-17 21:29:58,200 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-17 21:29:58,202 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:29:58,203 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:58,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1"]
2016-05-17 21:29:58,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:29:58,205 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:29:58,206 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:29:58,207 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-17 21:29:58,207 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:58,241 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:29:58,416 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:58,422 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:58,423 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:58,470 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2C3B5653E8454674B737D4ABB5FCE00D/A2D0F4A5F4F64E65979EE24006F321E8/.tez/application_1463520598422_0001 doesn't exist and is created
2016-05-17 21:29:58,559 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:58,559 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:58,560 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2C3B5653E8454674B737D4ABB5FCE00D/A2D0F4A5F4F64E65979EE24006F321E8/.tez/application_1463520598422_0001
2016-05-17 21:29:58,594 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520598422_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:58,642 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:58,659 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:58,660 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-17 21:29:58,660 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-17 21:29:58,660 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:58,687 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:58,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:58,695 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2C3B5653E8454674B737D4ABB5FCE00D/A2D0F4A5F4F64E65979EE24006F321E8/.tez/application_1463520598422_0001/localmode-log-dir
2016-05-17 21:29:58,695 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2C3B5653E8454674B737D4ABB5FCE00D/A2D0F4A5F4F64E65979EE24006F321E8/.tez/application_1463520598422_0001/localmode-log-dir/history.txt.appattempt_1463520598422_0001_000000, maxErrors=10
2016-05-17 21:29:58,696 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:58,698 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520598422_0001_000000, appSubmitTime=1463520598579, launchTime=1463520598593
2016-05-17 21:29:58,703 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:58,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:58,707 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:58,707 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:58,760 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 21:29:58,761 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 21:29:58,761 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:58,768 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41690
2016-05-17 21:29:58,838 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:58,861 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 21:29:58,861 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 21:29:58,861 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:58,905 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:58,905 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520598422_0001_000000, startTime=1463520598904
2016-05-17 21:29:58,905 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:58,961 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:58,977 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:58,977 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_'field2'_-1, applicationId=application_1463520598422_0001, dagName=[2C3B5653E8454674B737D4ABB5FCE00D/A2D0F4A5F4F64E65979EE24006F321E8] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-17 21:29:59,182 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2C3B5653E8454674B737D4ABB5FCE00D/A2D0F4A5F4F64E65979EE24006F321E8] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-17 21:29:59,281 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520598422_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2C3B5653E8454674B737D4ABB5FCE00D/A2D0F4A5F4F64E65979EE24006F321E8/.tez/application_1463520598422_0001/localmode-log-dir/dag_1463520598422_0001_1.dot
2016-05-17 21:29:59,284 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2C3B5653E8454674B737D4ABB5FCE00D/A2D0F4A5F4F64E65979EE24006F321E8/.tez/application_1463520598422_0001/localmode-log-dir/dag_1463520598422_0001_1-tez-dag.pb.txt
2016-05-17 21:29:59,371 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520598422_0001_1
2016-05-17 21:29:59,371 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:59,371 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2C3B5653E8454674B737D4ABB5FCE00D/A2D0F4A5F4F64E65979EE24006F321E8] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-17 21:29:59 Running Dag: dag_1463520598422_0001_1
2016-05-17 21:29:59,414 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:59,417 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520598422_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520598422_0001_1, submitTime=1463520599183
2016-05-17 21:29:59,447 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C8210EC222604A029325AEF4CCC3927C
2016-05-17 21:29:59,449 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:59,451 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:59,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520598422_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520598422_0001_1, initTime=1463520599419
2016-05-17 21:29:59,453 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520598422_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:59,457 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:59,457 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520598422_0001_1][Event:DAG_STARTED]: dagID=dag_1463520598422_0001_1, startTime=1463520599456
2016-05-17 21:29:59,457 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:59,460 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520598422_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:59,461 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C8210EC222604A029325AEF4CCC3927C : {109A12F76E904C38AB8D81B44F62B944={InputName=109A12F76E904C38AB8D81B44F62B944}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:59,462 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 109A12F76E904C38AB8D81B44F62B944, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:59,462 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C]
2016-05-17 21:29:59,472 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_'field2'_-1, applicationId=application_1463520598422_0001, dagName=[2C3B5653E8454674B737D4ABB5FCE00D/A2D0F4A5F4F64E65979EE24006F321E8] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-17 21:29:59,479 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C]
2016-05-17 21:29:59,479 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C]
2016-05-17 21:29:59,483 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C]. Starting root input initializers: 1
2016-05-17 21:29:59,500 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 109A12F76E904C38AB8D81B44F62B944 on vertex vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C]
2016-05-17 21:29:59,508 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520598422_0001, with dag id: dag_1463520598422_0001_1
2016-05-17 21:29:59,514 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:59,535 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:59,538 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 109A12F76E904C38AB8D81B44F62B944 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:59,540 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:59,540 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:59,585 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:59,588 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 109A12F76E904C38AB8D81B44F62B944 on vertex vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C]
2016-05-17 21:29:59,589 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:59,589 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {109A12F76E904C38AB8D81B44F62B944=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:59,611 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C] parallelism set to 1
2016-05-17 21:29:59,611 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:59,624 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520598422_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520598422_0001_1_00, eventCount=1
2016-05-17 21:29:59,628 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C]
2016-05-17 21:29:59,628 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=4537128D26604018BE915AC088C4B2F0, vertexId=vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:59,632 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=4537128D26604018BE915AC088C4B2F0, vertexId=vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C]
2016-05-17 21:29:59,651 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=4537128D26604018BE915AC088C4B2F0, vertexName=C8210EC222604A029325AEF4CCC3927C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:59,662 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=4537128D26604018BE915AC088C4B2F0, vertexId=vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C]
2016-05-17 21:29:59,667 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:59,667 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520598422_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C8210EC222604A029325AEF4CCC3927C, vertexId=vertex_1463520598422_0001_1_00, initRequestedTime=1463520599461, initedTime=1463520599666, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:59,668 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C] with distanceFromRoot: 0
2016-05-17 21:29:59,668 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:59,671 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C8210EC222604A029325AEF4CCC3927C
2016-05-17 21:29:59,671 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C]
2016-05-17 21:29:59,673 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:59,673 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520598422_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520598422_0001_1_00, startRequestedTime=1463520599669, startedTime=1463520599669
2016-05-17 21:29:59,674 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:59,701 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:59,701 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520598422_0001_1][Event:TASK_STARTED]: vertexName=C8210EC222604A029325AEF4CCC3927C, taskId=task_1463520598422_0001_1_00_000000, scheduledTime=1463520599700, launchTime=1463520599700
2016-05-17 21:29:59,702 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520598422_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:59,704 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C]
2016-05-17 21:29:59,707 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2C3B5653E8454674B737D4ABB5FCE00D/A2D0F4A5F4F64E65979EE24006F321E8] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1, VertexName: C8210EC222604A029325AEF4CCC3927C, VertexParallelism: 1, TaskAttemptID:attempt_1463520598422_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=109A12F76E904C38AB8D81B44F62B944, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=4537128D26604018BE915AC088C4B2F0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:59,710 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520598422_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:59,710 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:59,751 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:59,754 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:59,754 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:59,758 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520598422_0001_00_000001
2016-05-17 21:29:59,759 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520598422_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:59,775 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520598422_0001_00_000001
2016-05-17 21:29:59,776 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:59,782 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520598422_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:59,783 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:59,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520598422_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520598422_0001_00_000001, launchTime=1463520599783
2016-05-17 21:29:59,787 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520598422_0001_00_000001
2016-05-17 21:29:59,791 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520598422_0001_1_00_000000_0] to container: [container_1463520598422_0001_00_000001]
2016-05-17 21:29:59,792 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520598422_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:59,794 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520598422_0001_00_000001 given task: attempt_1463520598422_0001_1_00_000000_0
2016-05-17 21:29:59,794 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520598422_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520598422_0001_1_00_000000_0
2016-05-17 21:29:59 Starting to run new task attempt: attempt_1463520598422_0001_1_00_000000_0
2016-05-17 21:29:59,795 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520598422_0001_1_00_000000_0
2016-05-17 21:29:59,795 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520598422_0001_1_00_000000_0] started. Is using containerId: [container_1463520598422_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:59,796 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:59,797 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:59,797 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520598422_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C8210EC222604A029325AEF4CCC3927C, taskAttemptId=attempt_1463520598422_0001_1_00_000000_0, startTime=1463520599794, containerId=container_1463520598422_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520598422_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:59,798 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520598422_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:59,799 INFO counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters: GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-17 21:29:59,797 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:59,800 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:59,800 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:59,802 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:59,804 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520598422_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:59,827 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:59,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2C3B5653E8454674B737D4ABB5FCE00D/A2D0F4A5F4F64E65979EE24006F321E8] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1, VertexName: C8210EC222604A029325AEF4CCC3927C, VertexParallelism: 1, TaskAttemptID:attempt_1463520598422_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=109A12F76E904C38AB8D81B44F62B944, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=4537128D26604018BE915AC088C4B2F0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:59,829 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:59,835 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520598422_0001_1_00_000000_0
2016-05-17 21:29:59,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:59,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:59,845 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:59,845 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=4537128D26604018BE915AC088C4B2F0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:59,843 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520598422_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:59,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=109A12F76E904C38AB8D81B44F62B944, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:59,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:59,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 4537128D26604018BE915AC088C4B2F0
2016-05-17 21:29:59,848 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:59,849 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:59,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 109A12F76E904C38AB8D81B44F62B944
2016-05-17 21:29:59,851 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 109A12F76E904C38AB8D81B44F62B944
2016-05-17 21:29:59,851 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 4537128D26604018BE915AC088C4B2F0
2016-05-17 21:29:59,869 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:59,872 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:59,872 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 109A12F76E904C38AB8D81B44F62B944
2016-05-17 21:29:59,873 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 109A12F76E904C38AB8D81B44F62B944
2016-05-17 21:29:59,893 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C8210EC222604A029325AEF4CCC3927C, ordinal: 0
2016-05-17 21:29:59,894 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8210EC222604A029325AEF4CCC3927C, mem on start (mb), free: 197, total: 281, max: 672
2016-05-17 21:29:59,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:59,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:59,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:59,899 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:59,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 4537128D26604018BE915AC088C4B2F0
2016-05-17 21:29:59,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:59,900 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:59,908 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:59,909 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:59,909 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:59,909 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:59,910 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:59,910 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:29:59,910 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 109A12F76E904C38AB8D81B44F62B944, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:59,911 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 4537128D26604018BE915AC088C4B2F0, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:59,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:59,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 109A12F76E904C38AB8D81B44F62B944
2016-05-17 21:29:59,917 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:59,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 109A12F76E904C38AB8D81B44F62B944
2016-05-17 21:29:59,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 109A12F76E904C38AB8D81B44F62B944 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:59,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:59,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:59,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:59,920 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520598422_0001_1_00_000000_0
2016-05-17 21:29:59,922 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:59,932 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4c9befde
2016-05-17 21:29:59,945 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4c9befde
2016-05-17 21:29:59,946 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:59,946 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:59,966 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:59,990 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 109A12F76E904C38AB8D81B44F62B944
2016-05-17 21:29:59,990 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1"], id: 4537128D26604018BE915AC088C4B2F0
2016-05-17 21:29:59,991 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1"]
2016-05-17 21:29:59,994 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:59,994 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:59,997 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C8210EC222604A029325AEF4CCC3927C, all 1 inputs ready in: 00:00:00.002
2016-05-17 21:30:00,005 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:00,006 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:00,007 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520598422_0001_1_00_000000_0
2016-05-17 21:30:00,007 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520598422_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:00,009 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520598422_0001_1_00_000000_0
2016-05-17 21:30:00,009 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8210EC222604A029325AEF4CCC3927C, mem on close (mb), free: 190, total: 281, max: 672
2016-05-17 21:30:00 Completed running task attempt: attempt_1463520598422_0001_1_00_000000_0
2016-05-17 21:30:00,015 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520598422_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:00,019 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520599997, Process_Duration=12, Process_End_Time=1463520600009, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:00,019 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:00,021 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520598422_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C8210EC222604A029325AEF4CCC3927C, taskAttemptId=attempt_1463520598422_0001_1_00_000000_0, startTime=1463520599794, finishTime=1463520600016, timeTaken=222, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520599997, Process_Duration=12, Process_End_Time=1463520600009, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:00,023 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520598422_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:00,023 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:00,026 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:00,026 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:00,024 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520598422_0001_1_00_000000_0
2016-05-17 21:30:00,026 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520598422_0001_00_000001
2016-05-17 21:30:00,028 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:00,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520598422_0001_1][Event:TASK_FINISHED]: vertexName=C8210EC222604A029325AEF4CCC3927C, taskId=task_1463520598422_0001_1_00_000000, startTime=1463520599794, finishTime=1463520600027, timeTaken=233, status=SUCCEEDED, successfulAttemptID=attempt_1463520598422_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520599997, Process_Duration=12, Process_End_Time=1463520600009, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:00,035 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520598422_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:00,035 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C] : 1
2016-05-17 21:30:00,035 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C]
2016-05-17 21:30:00,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:00,047 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520598422_0001_1][Event:VERTEX_FINISHED]: vertexName=C8210EC222604A029325AEF4CCC3927C, vertexId=vertex_1463520598422_0001_1_00, initRequestedTime=1463520599461, initedTime=1463520599666, startRequestedTime=1463520599669, startedTime=1463520599669, finishTime=1463520600036, timeTaken=367, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520599997, Process_Duration=12, Process_End_Time=1463520600009, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520599794, firstTasksToStart=[ task_1463520598422_0001_1_00_000000 ], lastTaskFinishTime=1463520600016, lastTasksToFinish=[ task_1463520598422_0001_1_00_000000 ], minTaskDuration=222, maxTaskDuration=222, avgTaskDuration=222.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520598422_0001_1_00_000000 ], longestDurationTasks=[ task_1463520598422_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:00,047 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:00,048 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520598422_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:00,049 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:00,050 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520598422_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520598422_0001_00_000001, stoppedTime=1463520600049, exitStatus=0
2016-05-17 21:30:00,051 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520598422_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:00,051 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:00,052 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520598422_0001_1
2016-05-17 21:30:00,053 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520598422_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520598422_0001_1
2016-05-17 21:30:00,053 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 4537128D26604018BE915AC088C4B2F0 for vertex: vertex_1463520598422_0001_1_00 [C8210EC222604A029325AEF4CCC3927C]
2016-05-17 21:30:00,058 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520598422_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:00,058 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520598422_0001_00_000001: 31 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:00,058 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520598422_0001_00_000001, Exiting
2016-05-17 21:30:00,058 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520598422_0001_00_000001
2016-05-17 21:30:00,058 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520598422_0001_00_000001
2016-05-17 21:30:00,061 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520598422_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:00,061 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520598422_0001_00_000001 completed successfully
2016-05-17 21:30:00,073 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520598422_0001_1, queueSize=0
2016-05-17 21:30:00,073 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:00,076 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520598422_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520598422_0001_1, startTime=1463520599456, finishTime=1463520600066, timeTaken=610, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520599997, Process_Duration=12, Process_End_Time=1463520600009, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:00,078 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520598422_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:00,078 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520598422_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:00,079 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520598422_0001_00_000001
2016-05-17 21:30:00,080 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520598422_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:00 Completed Dag: dag_1463520598422_0001_1
2016-05-17 21:30:00,082 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520598422_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:00,082 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520598422_0001_1_post
2016-05-17 21:30:00,083 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:00,083 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:00,096 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.-1_'field2'_-1, applicationId=application_1463520598422_0001
2016-05-17 21:30:00,104 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:00,104 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:00,104 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:00,104 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:00,104 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:00,105 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:00,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:00,108 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:00,108 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:00,109 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:00,109 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:00,110 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:00,110 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:00,115 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2C3B5653E8454674B737D4ABB5FCE00D/A2D0F4A5F4F64E65979EE24006F321E8/.tez/application_1463520598422_0001
2016-05-17 21:30:00,117 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1/_temporary
2016-05-17 21:30:00,117 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:01.914
2016-05-17 21:30:00,217 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_-1
2016-05-17 21:30:00,278 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:00,278 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:00,392 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.063
2016-05-17 21:30:00,411 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.083
2016-05-17 21:30:00,412 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:00,413 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:00,413 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:00,433 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:00,434 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'
2016-05-17 21:30:00,460 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:00,460 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:00,460 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'"]
2016-05-17 21:30:00,460 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:00,461 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:00,461 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:00,461 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ULTS.-1_'field2'_'field2'
2016-05-17 21:30:00,462 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:00,477 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:00,483 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:00,483 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:00,483 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:00,494 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3A9F0EAA57DB4953978C3D2C0FA49823/ADDCC3BF663C45AEBE52E0C5C6C5FC09/.tez/application_1463520600483_0001 doesn't exist and is created
2016-05-17 21:30:00,529 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:00,530 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:00,532 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3A9F0EAA57DB4953978C3D2C0FA49823/ADDCC3BF663C45AEBE52E0C5C6C5FC09/.tez/application_1463520600483_0001
2016-05-17 21:30:00,536 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520600483_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:00,555 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:00,556 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:00,558 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:00,558 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:00,558 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3A9F0EAA57DB4953978C3D2C0FA49823/ADDCC3BF663C45AEBE52E0C5C6C5FC09/.tez/application_1463520600483_0001/localmode-log-dir
2016-05-17 21:30:00,559 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3A9F0EAA57DB4953978C3D2C0FA49823/ADDCC3BF663C45AEBE52E0C5C6C5FC09/.tez/application_1463520600483_0001/localmode-log-dir/history.txt.appattempt_1463520600483_0001_000000, maxErrors=10
2016-05-17 21:30:00,559 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:00,559 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520600483_0001_000000, appSubmitTime=1463520600535, launchTime=1463520600535
2016-05-17 21:30:00,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:00,576 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:00,578 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58790
2016-05-17 21:30:00,579 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:00,579 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:00,583 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:00,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:00,586 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520600483_0001_000000, startTime=1463520600586
2016-05-17 21:30:00,587 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:00,631 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:00,631 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:00,631 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ULTS.-1_'field2'_'field2', applicationId=application_1463520600483_0001, dagName=[3A9F0EAA57DB4953978C3D2C0FA49823/ADDCC3BF663C45AEBE52E0C5C6C5FC09] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-17 21:30:00,659 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3A9F0EAA57DB4953978C3D2C0FA49823/ADDCC3BF663C45AEBE52E0C5C6C5FC09] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-17 21:30:00,662 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520600483_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3A9F0EAA57DB4953978C3D2C0FA49823/ADDCC3BF663C45AEBE52E0C5C6C5FC09/.tez/application_1463520600483_0001/localmode-log-dir/dag_1463520600483_0001_1.dot
2016-05-17 21:30:00,665 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3A9F0EAA57DB4953978C3D2C0FA49823/ADDCC3BF663C45AEBE52E0C5C6C5FC09/.tez/application_1463520600483_0001/localmode-log-dir/dag_1463520600483_0001_1-tez-dag.pb.txt
2016-05-17 21:30:00,691 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520600483_0001_1
2016-05-17 21:30:00,691 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:00,691 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3A9F0EAA57DB4953978C3D2C0FA49823/ADDCC3BF663C45AEBE52E0C5C6C5FC09] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-17 21:30:00 Running Dag: dag_1463520600483_0001_1
2016-05-17 21:30:00,706 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520600483_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520600483_0001_1, submitTime=1463520600659
2016-05-17 21:30:00,706 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:00,707 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B857FBB3C49A4BA1941749201784F5E9
2016-05-17 21:30:00,708 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:00,708 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520600483_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520600483_0001_1, initTime=1463520600706
2016-05-17 21:30:00,708 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:00,708 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520600483_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:00,709 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:00,709 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520600483_0001_1][Event:DAG_STARTED]: dagID=dag_1463520600483_0001_1, startTime=1463520600709
2016-05-17 21:30:00,710 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:00,710 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520600483_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:00,710 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B857FBB3C49A4BA1941749201784F5E9 : {F9D23B18339E4588BB3CFDF05C66AADF={InputName=F9D23B18339E4588BB3CFDF05C66AADF}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:00,710 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F9D23B18339E4588BB3CFDF05C66AADF, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:00,710 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9]
2016-05-17 21:30:00,713 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9]
2016-05-17 21:30:00,713 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9]
2016-05-17 21:30:00,713 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9]. Starting root input initializers: 1
2016-05-17 21:30:00,713 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ULTS.-1_'field2'_'field2', applicationId=application_1463520600483_0001, dagName=[3A9F0EAA57DB4953978C3D2C0FA49823/ADDCC3BF663C45AEBE52E0C5C6C5FC09] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-17 21:30:00,713 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:00,714 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F9D23B18339E4588BB3CFDF05C66AADF on vertex vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9]
2016-05-17 21:30:00,717 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:00,718 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F9D23B18339E4588BB3CFDF05C66AADF asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:00,718 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:00,719 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:00,720 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:00,721 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F9D23B18339E4588BB3CFDF05C66AADF on vertex vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9]
2016-05-17 21:30:00,721 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:00,721 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F9D23B18339E4588BB3CFDF05C66AADF=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:00,722 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9] parallelism set to 1
2016-05-17 21:30:00,722 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:00,722 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520600483_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520600483_0001_1_00, eventCount=1
2016-05-17 21:30:00,722 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9]
2016-05-17 21:30:00,724 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A5974ACB717B4681AAD0754B1C93C0A0, vertexId=vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:00,724 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A5974ACB717B4681AAD0754B1C93C0A0, vertexId=vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9]
2016-05-17 21:30:00,728 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A5974ACB717B4681AAD0754B1C93C0A0, vertexName=B857FBB3C49A4BA1941749201784F5E9, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:00,728 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A5974ACB717B4681AAD0754B1C93C0A0, vertexId=vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9]
2016-05-17 21:30:00,730 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:00,730 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520600483_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B857FBB3C49A4BA1941749201784F5E9, vertexId=vertex_1463520600483_0001_1_00, initRequestedTime=1463520600710, initedTime=1463520600730, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:00,730 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9] with distanceFromRoot: 0
2016-05-17 21:30:00,731 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:00,731 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B857FBB3C49A4BA1941749201784F5E9
2016-05-17 21:30:00,731 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9]
2016-05-17 21:30:00,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:00,732 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520600483_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520600483_0001_1_00, startRequestedTime=1463520600731, startedTime=1463520600731
2016-05-17 21:30:00,732 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:00,733 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:00,733 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520600483_0001_1][Event:TASK_STARTED]: vertexName=B857FBB3C49A4BA1941749201784F5E9, taskId=task_1463520600483_0001_1_00_000000, scheduledTime=1463520600733, launchTime=1463520600733
2016-05-17 21:30:00,733 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520600483_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:00,734 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9]
2016-05-17 21:30:00,734 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3A9F0EAA57DB4953978C3D2C0FA49823/ADDCC3BF663C45AEBE52E0C5C6C5FC09] every.every/(1/1) ...ULTS.-1_'field2'_'field2', VertexName: B857FBB3C49A4BA1941749201784F5E9, VertexParallelism: 1, TaskAttemptID:attempt_1463520600483_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9D23B18339E4588BB3CFDF05C66AADF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A5974ACB717B4681AAD0754B1C93C0A0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:00,735 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520600483_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:00,735 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:00,736 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:00,736 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:00,736 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:00,737 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520600483_0001_00_000001
2016-05-17 21:30:00,737 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520600483_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:00,743 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520600483_0001_00_000001
2016-05-17 21:30:00,743 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:00,744 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520600483_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:00,745 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:00,746 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520600483_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520600483_0001_00_000001, launchTime=1463520600745
2016-05-17 21:30:00,746 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520600483_0001_00_000001
2016-05-17 21:30:00,747 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520600483_0001, with dag id: dag_1463520600483_0001_1
2016-05-17 21:30:00,748 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520600483_0001_1_00_000000_0] to container: [container_1463520600483_0001_00_000001]
2016-05-17 21:30:00,749 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520600483_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:00,750 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520600483_0001_1_00_000000_0] started. Is using containerId: [container_1463520600483_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:00,751 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520600483_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B857FBB3C49A4BA1941749201784F5E9, taskAttemptId=attempt_1463520600483_0001_1_00_000000_0, startTime=1463520600750, containerId=container_1463520600483_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520600483_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:00,751 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520600483_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:00,751 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:00,751 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520600483_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:00,753 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520600483_0001_00_000001 given task: attempt_1463520600483_0001_1_00_000000_0
2016-05-17 21:30:00,753 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520600483_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520600483_0001_1_00_000000_0
2016-05-17 21:30:00 Starting to run new task attempt: attempt_1463520600483_0001_1_00_000000_0
2016-05-17 21:30:00,754 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520600483_0001_1_00_000000_0
2016-05-17 21:30:00,755 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:00,755 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:00,755 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:00,755 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:00,755 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:00,756 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:00,756 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3A9F0EAA57DB4953978C3D2C0FA49823/ADDCC3BF663C45AEBE52E0C5C6C5FC09] every.every/(1/1) ...ULTS.-1_'field2'_'field2', VertexName: B857FBB3C49A4BA1941749201784F5E9, VertexParallelism: 1, TaskAttemptID:attempt_1463520600483_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9D23B18339E4588BB3CFDF05C66AADF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A5974ACB717B4681AAD0754B1C93C0A0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:00,756 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:00,760 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520600483_0001_1_00_000000_0
2016-05-17 21:30:00,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:00,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:00,763 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520600483_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:00,764 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F9D23B18339E4588BB3CFDF05C66AADF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:00,764 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:00,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:00,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F9D23B18339E4588BB3CFDF05C66AADF
2016-05-17 21:30:00,770 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:00,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A5974ACB717B4681AAD0754B1C93C0A0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:00,770 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F9D23B18339E4588BB3CFDF05C66AADF
2016-05-17 21:30:00,771 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:00,771 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A5974ACB717B4681AAD0754B1C93C0A0
2016-05-17 21:30:00,772 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:00,772 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A5974ACB717B4681AAD0754B1C93C0A0
2016-05-17 21:30:00,788 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:00,788 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F9D23B18339E4588BB3CFDF05C66AADF
2016-05-17 21:30:00,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F9D23B18339E4588BB3CFDF05C66AADF
2016-05-17 21:30:00,789 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B857FBB3C49A4BA1941749201784F5E9, ordinal: 0
2016-05-17 21:30:00,789 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B857FBB3C49A4BA1941749201784F5E9, mem on start (mb), free: 319, total: 397, max: 672
2016-05-17 21:30:00,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:00,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:00,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:00,796 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:00,796 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A5974ACB717B4681AAD0754B1C93C0A0
2016-05-17 21:30:00,796 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:00,796 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:00,796 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:00,799 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:00,799 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:00,799 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:00,800 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:00,800 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:00,800 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F9D23B18339E4588BB3CFDF05C66AADF, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:00,800 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A5974ACB717B4681AAD0754B1C93C0A0, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:00,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:00,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F9D23B18339E4588BB3CFDF05C66AADF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:00,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F9D23B18339E4588BB3CFDF05C66AADF
2016-05-17 21:30:00,801 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:00,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F9D23B18339E4588BB3CFDF05C66AADF
2016-05-17 21:30:00,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:00,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:00,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:00,803 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520600483_0001_1_00_000000_0
2016-05-17 21:30:00,803 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:00,809 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7c67ea36
2016-05-17 21:30:00,811 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:00,812 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7c67ea36
2016-05-17 21:30:00,812 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:00,812 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:00,813 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F9D23B18339E4588BB3CFDF05C66AADF
2016-05-17 21:30:00,813 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'"], id: A5974ACB717B4681AAD0754B1C93C0A0
2016-05-17 21:30:00,813 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'"]
2016-05-17 21:30:00,813 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:00,814 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:00,814 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B857FBB3C49A4BA1941749201784F5E9, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:00,817 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:00,817 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:00,818 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520600483_0001_1_00_000000_0
2016-05-17 21:30:00,818 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520600483_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:00,819 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520600483_0001_1_00_000000_0
2016-05-17 21:30:00,819 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B857FBB3C49A4BA1941749201784F5E9, mem on close (mb), free: 316, total: 397, max: 672
2016-05-17 21:30:00 Completed running task attempt: attempt_1463520600483_0001_1_00_000000_0
2016-05-17 21:30:00,821 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520600483_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:00,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520600483_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B857FBB3C49A4BA1941749201784F5E9, taskAttemptId=attempt_1463520600483_0001_1_00_000000_0, startTime=1463520600750, finishTime=1463520600822, timeTaken=72, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520600814, Process_Duration=5, Process_End_Time=1463520600819, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:00,826 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520600483_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:00,826 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:00,826 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520600483_0001_1_00_000000_0
2016-05-17 21:30:00,828 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:00,829 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520600483_0001_1][Event:TASK_FINISHED]: vertexName=B857FBB3C49A4BA1941749201784F5E9, taskId=task_1463520600483_0001_1_00_000000, startTime=1463520600750, finishTime=1463520600828, timeTaken=78, status=SUCCEEDED, successfulAttemptID=attempt_1463520600483_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520600814, Process_Duration=5, Process_End_Time=1463520600819, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:00,829 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:00,829 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520600483_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:00,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520600814, Process_Duration=5, Process_End_Time=1463520600819, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:00,831 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9] : 1
2016-05-17 21:30:00,831 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:00,831 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9]
2016-05-17 21:30:00,832 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:00,832 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520600483_0001_00_000001
2016-05-17 21:30:00,833 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:00,834 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520600483_0001_1][Event:VERTEX_FINISHED]: vertexName=B857FBB3C49A4BA1941749201784F5E9, vertexId=vertex_1463520600483_0001_1_00, initRequestedTime=1463520600710, initedTime=1463520600730, startRequestedTime=1463520600731, startedTime=1463520600731, finishTime=1463520600832, timeTaken=101, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520600814, Process_Duration=5, Process_End_Time=1463520600819, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520600750, firstTasksToStart=[ task_1463520600483_0001_1_00_000000 ], lastTaskFinishTime=1463520600822, lastTasksToFinish=[ task_1463520600483_0001_1_00_000000 ], minTaskDuration=72, maxTaskDuration=72, avgTaskDuration=72.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520600483_0001_1_00_000000 ], longestDurationTasks=[ task_1463520600483_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:00,834 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:00,834 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520600483_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:00,835 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520600483_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520600483_0001_00_000001, stoppedTime=1463520600834, exitStatus=0
2016-05-17 21:30:00,835 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:00,835 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520600483_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:00,836 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:00,836 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520600483_0001_1
2016-05-17 21:30:00,836 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520600483_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520600483_0001_1
2016-05-17 21:30:00,836 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A5974ACB717B4681AAD0754B1C93C0A0 for vertex: vertex_1463520600483_0001_1_00 [B857FBB3C49A4BA1941749201784F5E9]
2016-05-17 21:30:00,844 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520600483_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:00,844 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520600483_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:00,845 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520600483_0001_00_000001, Exiting
2016-05-17 21:30:00,845 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520600483_0001_00_000001
2016-05-17 21:30:00,845 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520600483_0001_00_000001
2016-05-17 21:30:00,847 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520600483_0001_1, queueSize=0
2016-05-17 21:30:00,848 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:00,848 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520600483_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:00,850 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520600483_0001_00_000001 completed successfully
2016-05-17 21:30:00,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520600483_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520600483_0001_1, startTime=1463520600709, finishTime=1463520600844, timeTaken=135, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520600814, Process_Duration=5, Process_End_Time=1463520600819, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:00,851 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520600483_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:00,851 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520600483_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:00,851 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520600483_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:00,851 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520600483_0001_00_000001
2016-05-17 21:30:00 Completed Dag: dag_1463520600483_0001_1
2016-05-17 21:30:00,852 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520600483_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:00,852 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520600483_0001_1_post
2016-05-17 21:30:00,852 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:00,852 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:00,861 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ULTS.-1_'field2'_'field2', applicationId=application_1463520600483_0001
2016-05-17 21:30:00,863 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:00,863 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:00,864 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:00,864 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:00,864 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:00,864 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:00,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:00,865 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:00,865 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:00,865 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:00,865 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:00,866 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:00,866 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:00,871 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3A9F0EAA57DB4953978C3D2C0FA49823/ADDCC3BF663C45AEBE52E0C5C6C5FC09/.tez/application_1463520600483_0001
2016-05-17 21:30:00,872 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'/_temporary
2016-05-17 21:30:00,872 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.412
2016-05-17 21:30:00,961 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_'field2'
2016-05-17 21:30:01,002 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:01,003 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:01,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.035
2016-05-17 21:30:01,089 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.044
2016-05-17 21:30:01,090 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:01,090 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:01,091 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:01,109 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:01,110 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'
2016-05-17 21:30:01,131 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:01,132 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:01,132 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'"]
2016-05-17 21:30:01,132 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:01,132 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:01,133 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:01,133 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...SULTS.-1_'field2'_'field'
2016-05-17 21:30:01,134 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:01,149 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:01,156 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:01,156 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:01,157 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:01,166 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5D6F5D4CA7C8424391F47FEB94BBE49F/F86202729DF4424D8827D7BFFB60E6D2/.tez/application_1463520601156_0001 doesn't exist and is created
2016-05-17 21:30:01,198 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:01,198 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:01,199 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5D6F5D4CA7C8424391F47FEB94BBE49F/F86202729DF4424D8827D7BFFB60E6D2/.tez/application_1463520601156_0001
2016-05-17 21:30:01,206 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520601156_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:01,219 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:01,220 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:01,222 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:01,222 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:01,223 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5D6F5D4CA7C8424391F47FEB94BBE49F/F86202729DF4424D8827D7BFFB60E6D2/.tez/application_1463520601156_0001/localmode-log-dir
2016-05-17 21:30:01,223 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5D6F5D4CA7C8424391F47FEB94BBE49F/F86202729DF4424D8827D7BFFB60E6D2/.tez/application_1463520601156_0001/localmode-log-dir/history.txt.appattempt_1463520601156_0001_000000, maxErrors=10
2016-05-17 21:30:01,223 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:01,224 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520601156_0001_000000, appSubmitTime=1463520601205, launchTime=1463520601205
2016-05-17 21:30:01,229 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:01,234 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:01,237 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:37641
2016-05-17 21:30:01,240 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:01,240 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:01,241 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:01,244 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:01,244 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520601156_0001_000000, startTime=1463520601244
2016-05-17 21:30:01,244 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:01,299 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:01,299 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:01,299 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_'field', applicationId=application_1463520601156_0001, dagName=[5D6F5D4CA7C8424391F47FEB94BBE49F/F86202729DF4424D8827D7BFFB60E6D2] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-17 21:30:01,322 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5D6F5D4CA7C8424391F47FEB94BBE49F/F86202729DF4424D8827D7BFFB60E6D2] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-17 21:30:01,324 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520601156_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5D6F5D4CA7C8424391F47FEB94BBE49F/F86202729DF4424D8827D7BFFB60E6D2/.tez/application_1463520601156_0001/localmode-log-dir/dag_1463520601156_0001_1.dot
2016-05-17 21:30:01,325 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5D6F5D4CA7C8424391F47FEB94BBE49F/F86202729DF4424D8827D7BFFB60E6D2/.tez/application_1463520601156_0001/localmode-log-dir/dag_1463520601156_0001_1-tez-dag.pb.txt
2016-05-17 21:30:01,351 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520601156_0001_1
2016-05-17 21:30:01,351 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:01,351 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5D6F5D4CA7C8424391F47FEB94BBE49F/F86202729DF4424D8827D7BFFB60E6D2] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-17 21:30:01 Running Dag: dag_1463520601156_0001_1
2016-05-17 21:30:01,366 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:01,366 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601156_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520601156_0001_1, submitTime=1463520601322
2016-05-17 21:30:01,367 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A8198E4579B849D2A41413367CF857A8
2016-05-17 21:30:01,368 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:01,368 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:01,368 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601156_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520601156_0001_1, initTime=1463520601367
2016-05-17 21:30:01,370 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520601156_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:01,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601156_0001_1][Event:DAG_STARTED]: dagID=dag_1463520601156_0001_1, startTime=1463520601370
2016-05-17 21:30:01,371 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:01,371 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:01,371 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520601156_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:01,371 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A8198E4579B849D2A41413367CF857A8 : {B1050139239F4BC4A3BE1D879911F94F={InputName=B1050139239F4BC4A3BE1D879911F94F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:01,371 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B1050139239F4BC4A3BE1D879911F94F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:01,372 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8]
2016-05-17 21:30:01,372 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8]
2016-05-17 21:30:01,372 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8]
2016-05-17 21:30:01,372 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8]. Starting root input initializers: 1
2016-05-17 21:30:01,373 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_'field', applicationId=application_1463520601156_0001, dagName=[5D6F5D4CA7C8424391F47FEB94BBE49F/F86202729DF4424D8827D7BFFB60E6D2] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-17 21:30:01,373 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:01,373 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B1050139239F4BC4A3BE1D879911F94F on vertex vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8]
2016-05-17 21:30:01,376 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:01,376 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B1050139239F4BC4A3BE1D879911F94F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:01,377 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:01,378 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:01,380 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:01,382 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B1050139239F4BC4A3BE1D879911F94F on vertex vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8]
2016-05-17 21:30:01,382 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:01,382 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B1050139239F4BC4A3BE1D879911F94F=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:01,383 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8] parallelism set to 1
2016-05-17 21:30:01,383 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:01,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601156_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520601156_0001_1_00, eventCount=1
2016-05-17 21:30:01,385 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8]
2016-05-17 21:30:01,386 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=721DDB394B71464BA0126967D5CC114C, vertexId=vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:01,386 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=721DDB394B71464BA0126967D5CC114C, vertexId=vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8]
2016-05-17 21:30:01,390 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=721DDB394B71464BA0126967D5CC114C, vertexName=A8198E4579B849D2A41413367CF857A8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:01,390 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=721DDB394B71464BA0126967D5CC114C, vertexId=vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8]
2016-05-17 21:30:01,393 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:01,393 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601156_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A8198E4579B849D2A41413367CF857A8, vertexId=vertex_1463520601156_0001_1_00, initRequestedTime=1463520601371, initedTime=1463520601392, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:01,393 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8] with distanceFromRoot: 0
2016-05-17 21:30:01,393 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:01,393 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A8198E4579B849D2A41413367CF857A8
2016-05-17 21:30:01,394 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8]
2016-05-17 21:30:01,394 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:01,394 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601156_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520601156_0001_1_00, startRequestedTime=1463520601393, startedTime=1463520601393
2016-05-17 21:30:01,394 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:01,395 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:01,395 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601156_0001_1][Event:TASK_STARTED]: vertexName=A8198E4579B849D2A41413367CF857A8, taskId=task_1463520601156_0001_1_00_000000, scheduledTime=1463520601395, launchTime=1463520601395
2016-05-17 21:30:01,395 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520601156_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:01,396 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8]
2016-05-17 21:30:01,396 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5D6F5D4CA7C8424391F47FEB94BBE49F/F86202729DF4424D8827D7BFFB60E6D2] every.every/(1/1) ...SULTS.-1_'field2'_'field', VertexName: A8198E4579B849D2A41413367CF857A8, VertexParallelism: 1, TaskAttemptID:attempt_1463520601156_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B1050139239F4BC4A3BE1D879911F94F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=721DDB394B71464BA0126967D5CC114C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:01,396 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520601156_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:01,397 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:01,398 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:01,398 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:01,399 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:01,399 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520601156_0001_00_000001
2016-05-17 21:30:01,400 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520601156_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:01,401 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520601156_0001_00_000001
2016-05-17 21:30:01,402 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:01,402 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520601156_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:01,403 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:01,403 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601156_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520601156_0001_00_000001, launchTime=1463520601403
2016-05-17 21:30:01,404 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520601156_0001_00_000001
2016-05-17 21:30:01,406 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520601156_0001, with dag id: dag_1463520601156_0001_1
2016-05-17 21:30:01,407 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520601156_0001_1_00_000000_0] to container: [container_1463520601156_0001_00_000001]
2016-05-17 21:30:01,408 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520601156_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:01,409 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520601156_0001_00_000001 given task: attempt_1463520601156_0001_1_00_000000_0
2016-05-17 21:30:01,409 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520601156_0001_1_00_000000_0] started. Is using containerId: [container_1463520601156_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:01,409 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601156_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A8198E4579B849D2A41413367CF857A8, taskAttemptId=attempt_1463520601156_0001_1_00_000000_0, startTime=1463520601409, containerId=container_1463520601156_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520601156_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:01,409 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520601156_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:01,410 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520601156_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:01,410 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:01,409 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520601156_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520601156_0001_1_00_000000_0
2016-05-17 21:30:01 Starting to run new task attempt: attempt_1463520601156_0001_1_00_000000_0
2016-05-17 21:30:01,411 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520601156_0001_1_00_000000_0
2016-05-17 21:30:01,411 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:01,412 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:01,412 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:01,412 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:01,412 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:01,412 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:01,412 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5D6F5D4CA7C8424391F47FEB94BBE49F/F86202729DF4424D8827D7BFFB60E6D2] every.every/(1/1) ...SULTS.-1_'field2'_'field', VertexName: A8198E4579B849D2A41413367CF857A8, VertexParallelism: 1, TaskAttemptID:attempt_1463520601156_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B1050139239F4BC4A3BE1D879911F94F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=721DDB394B71464BA0126967D5CC114C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:01,412 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:01,413 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520601156_0001_1_00_000000_0
2016-05-17 21:30:01,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:01,415 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:01,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B1050139239F4BC4A3BE1D879911F94F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:01,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:01,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:01,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B1050139239F4BC4A3BE1D879911F94F
2016-05-17 21:30:01,417 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B1050139239F4BC4A3BE1D879911F94F
2016-05-17 21:30:01,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=721DDB394B71464BA0126967D5CC114C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:01,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:01,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 721DDB394B71464BA0126967D5CC114C
2016-05-17 21:30:01,424 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:01,424 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 721DDB394B71464BA0126967D5CC114C
2016-05-17 21:30:01,426 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:01,426 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B1050139239F4BC4A3BE1D879911F94F
2016-05-17 21:30:01,426 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:01,426 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520601156_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:01,427 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B1050139239F4BC4A3BE1D879911F94F
2016-05-17 21:30:01,441 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:01,441 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 721DDB394B71464BA0126967D5CC114C
2016-05-17 21:30:01,450 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A8198E4579B849D2A41413367CF857A8, ordinal: 0
2016-05-17 21:30:01,450 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A8198E4579B849D2A41413367CF857A8, mem on start (mb), free: 231, total: 397, max: 672
2016-05-17 21:30:01,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:01,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:01,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:01,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:01,451 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:01,452 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:01,453 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:01,454 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:01,454 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:01,455 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:01,455 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:01,455 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B1050139239F4BC4A3BE1D879911F94F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:01,456 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 721DDB394B71464BA0126967D5CC114C, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:01,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:01,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B1050139239F4BC4A3BE1D879911F94F
2016-05-17 21:30:01,457 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:01,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B1050139239F4BC4A3BE1D879911F94F
2016-05-17 21:30:01,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B1050139239F4BC4A3BE1D879911F94F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:01,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:01,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:01,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:01,459 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520601156_0001_1_00_000000_0
2016-05-17 21:30:01,459 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:01,462 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7c2d50f7
2016-05-17 21:30:01,466 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:01,467 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7c2d50f7
2016-05-17 21:30:01,467 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:01,467 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:01,468 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B1050139239F4BC4A3BE1D879911F94F
2016-05-17 21:30:01,468 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'"], id: 721DDB394B71464BA0126967D5CC114C
2016-05-17 21:30:01,469 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'"]
2016-05-17 21:30:01,469 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:01,470 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:01,470 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A8198E4579B849D2A41413367CF857A8, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:01,474 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:01,475 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:01,476 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520601156_0001_1_00_000000_0
2016-05-17 21:30:01,476 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520601156_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:01,477 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520601156_0001_1_00_000000_0
2016-05-17 21:30:01,477 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A8198E4579B849D2A41413367CF857A8, mem on close (mb), free: 230, total: 397, max: 672
2016-05-17 21:30:01 Completed running task attempt: attempt_1463520601156_0001_1_00_000000_0
2016-05-17 21:30:01,479 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520601156_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:01,481 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520601470, Process_Duration=7, Process_End_Time=1463520601477, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:01,482 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:01,482 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601156_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A8198E4579B849D2A41413367CF857A8, taskAttemptId=attempt_1463520601156_0001_1_00_000000_0, startTime=1463520601409, finishTime=1463520601481, timeTaken=72, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520601470, Process_Duration=7, Process_End_Time=1463520601477, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:01,483 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520601156_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:01,483 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520601156_0001_1_00_000000_0
2016-05-17 21:30:01,484 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601156_0001_1][Event:TASK_FINISHED]: vertexName=A8198E4579B849D2A41413367CF857A8, taskId=task_1463520601156_0001_1_00_000000, startTime=1463520601409, finishTime=1463520601483, timeTaken=74, status=SUCCEEDED, successfulAttemptID=attempt_1463520601156_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520601470, Process_Duration=7, Process_End_Time=1463520601477, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:01,484 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520601156_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:01,484 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8] : 1
2016-05-17 21:30:01,484 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8]
2016-05-17 21:30:01,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:01,485 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:01,488 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:01,488 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:01,489 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520601156_0001_00_000001
2016-05-17 21:30:01,489 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601156_0001_1][Event:VERTEX_FINISHED]: vertexName=A8198E4579B849D2A41413367CF857A8, vertexId=vertex_1463520601156_0001_1_00, initRequestedTime=1463520601371, initedTime=1463520601392, startRequestedTime=1463520601393, startedTime=1463520601393, finishTime=1463520601484, timeTaken=91, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520601470, Process_Duration=7, Process_End_Time=1463520601477, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520601409, firstTasksToStart=[ task_1463520601156_0001_1_00_000000 ], lastTaskFinishTime=1463520601481, lastTasksToFinish=[ task_1463520601156_0001_1_00_000000 ], minTaskDuration=72, maxTaskDuration=72, avgTaskDuration=72.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520601156_0001_1_00_000000 ], longestDurationTasks=[ task_1463520601156_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:01,490 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:01,490 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:01,490 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520601156_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:01,491 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601156_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520601156_0001_00_000001, stoppedTime=1463520601491, exitStatus=0
2016-05-17 21:30:01,492 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520601156_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:01,492 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:01,492 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520601156_0001_1
2016-05-17 21:30:01,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601156_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520601156_0001_1
2016-05-17 21:30:01,492 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:01,492 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 721DDB394B71464BA0126967D5CC114C for vertex: vertex_1463520601156_0001_1_00 [A8198E4579B849D2A41413367CF857A8]
2016-05-17 21:30:01,501 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520601156_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:01,501 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520601156_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:01,501 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520601156_0001_00_000001, Exiting
2016-05-17 21:30:01,501 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520601156_0001_00_000001
2016-05-17 21:30:01,502 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520601156_0001_00_000001
2016-05-17 21:30:01,506 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520601156_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:01,506 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520601156_0001_00_000001 completed successfully
2016-05-17 21:30:01,510 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520601156_0001_1, queueSize=0
2016-05-17 21:30:01,510 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:01,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601156_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520601156_0001_1, startTime=1463520601370, finishTime=1463520601508, timeTaken=138, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520601470, Process_Duration=7, Process_End_Time=1463520601477, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:01,512 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520601156_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:01,513 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520601156_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:01,513 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520601156_0001_00_000001
2016-05-17 21:30:01,514 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520601156_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:01 Completed Dag: dag_1463520601156_0001_1
2016-05-17 21:30:01,516 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520601156_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:01,516 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520601156_0001_1_post
2016-05-17 21:30:01,516 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:01,516 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:01,522 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...SULTS.-1_'field2'_'field', applicationId=application_1463520601156_0001
2016-05-17 21:30:01,525 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:01,525 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:01,526 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:01,526 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:01,526 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:01,527 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:01,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:01,528 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:01,528 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:01,528 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:01,529 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:01,529 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:01,530 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:01,537 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5D6F5D4CA7C8424391F47FEB94BBE49F/F86202729DF4424D8827D7BFFB60E6D2/.tez/application_1463520601156_0001
2016-05-17 21:30:01,537 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'/_temporary
2016-05-17 21:30:01,538 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.406
2016-05-17 21:30:01,644 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_'field'
2016-05-17 21:30:01,678 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:01,678 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:01,741 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-17 21:30:01,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.039
2016-05-17 21:30:01,758 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:01,758 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:01,758 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:01,783 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:01,783 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS
2016-05-17 21:30:01,809 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:01,809 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:01,810 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS"]
2016-05-17 21:30:01,810 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:01,810 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:01,810 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:01,811 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-17 21:30:01,812 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:01,834 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:01,839 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:01,840 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:01,840 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:01,851 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/75C99928FD4F4A40818F88BA1E5CCADF/C7A8D603E5434A538BFF1465F382B824/.tez/application_1463520601840_0001 doesn't exist and is created
2016-05-17 21:30:01,871 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:01,871 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:01,872 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/75C99928FD4F4A40818F88BA1E5CCADF/C7A8D603E5434A538BFF1465F382B824/.tez/application_1463520601840_0001
2016-05-17 21:30:01,875 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520601840_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:01,894 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:01,895 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:01,897 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:01,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:01,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/75C99928FD4F4A40818F88BA1E5CCADF/C7A8D603E5434A538BFF1465F382B824/.tez/application_1463520601840_0001/localmode-log-dir
2016-05-17 21:30:01,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/75C99928FD4F4A40818F88BA1E5CCADF/C7A8D603E5434A538BFF1465F382B824/.tez/application_1463520601840_0001/localmode-log-dir/history.txt.appattempt_1463520601840_0001_000000, maxErrors=10
2016-05-17 21:30:01,898 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:01,899 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520601840_0001_000000, appSubmitTime=1463520601875, launchTime=1463520601875
2016-05-17 21:30:01,910 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:01,914 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:01,914 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:60428
2016-05-17 21:30:01,915 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:01,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:01,917 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:01,919 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520601840_0001_000000, startTime=1463520601919
2016-05-17 21:30:01,919 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:01,920 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:01,971 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:01,971 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:01,971 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_RESULTS, applicationId=application_1463520601840_0001, dagName=[75C99928FD4F4A40818F88BA1E5CCADF/C7A8D603E5434A538BFF1465F382B824] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-17 21:30:01,997 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [75C99928FD4F4A40818F88BA1E5CCADF/C7A8D603E5434A538BFF1465F382B824] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-17 21:30:01,999 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520601840_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/75C99928FD4F4A40818F88BA1E5CCADF/C7A8D603E5434A538BFF1465F382B824/.tez/application_1463520601840_0001/localmode-log-dir/dag_1463520601840_0001_1.dot
2016-05-17 21:30:02,000 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/75C99928FD4F4A40818F88BA1E5CCADF/C7A8D603E5434A538BFF1465F382B824/.tez/application_1463520601840_0001/localmode-log-dir/dag_1463520601840_0001_1-tez-dag.pb.txt
2016-05-17 21:30:02,033 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520601840_0001_1
2016-05-17 21:30:02,034 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:02,034 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [75C99928FD4F4A40818F88BA1E5CCADF/C7A8D603E5434A538BFF1465F382B824] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-17 21:30:02 Running Dag: dag_1463520601840_0001_1
2016-05-17 21:30:02,049 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601840_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520601840_0001_1, submitTime=1463520601997
2016-05-17 21:30:02,049 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:02,050 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D5D222F5273044BD9FB8043377CC2954
2016-05-17 21:30:02,050 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:02,050 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601840_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520601840_0001_1, initTime=1463520602049
2016-05-17 21:30:02,050 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:02,052 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520601840_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:02,053 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:02,053 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601840_0001_1][Event:DAG_STARTED]: dagID=dag_1463520601840_0001_1, startTime=1463520602053
2016-05-17 21:30:02,053 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:02,053 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520601840_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:02,053 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_RESULTS, applicationId=application_1463520601840_0001, dagName=[75C99928FD4F4A40818F88BA1E5CCADF/C7A8D603E5434A538BFF1465F382B824] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-17 21:30:02,054 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D5D222F5273044BD9FB8043377CC2954 : {43AB75A595214468826EDE0D9CEF2476={InputName=43AB75A595214468826EDE0D9CEF2476}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:02,054 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 43AB75A595214468826EDE0D9CEF2476, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:02,054 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954]
2016-05-17 21:30:02,054 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954]
2016-05-17 21:30:02,054 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954]
2016-05-17 21:30:02,055 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954]. Starting root input initializers: 1
2016-05-17 21:30:02,055 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:02,056 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 43AB75A595214468826EDE0D9CEF2476 on vertex vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954]
2016-05-17 21:30:02,060 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:02,061 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 43AB75A595214468826EDE0D9CEF2476 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:02,061 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:02,062 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:02,064 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:02,064 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 43AB75A595214468826EDE0D9CEF2476 on vertex vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954]
2016-05-17 21:30:02,064 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:02,064 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {43AB75A595214468826EDE0D9CEF2476=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:02,065 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954] parallelism set to 1
2016-05-17 21:30:02,065 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:02,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601840_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520601840_0001_1_00, eventCount=1
2016-05-17 21:30:02,065 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954]
2016-05-17 21:30:02,065 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=C6347BDEEF1D47C4AA79FA7BDEB8754E, vertexId=vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:02,066 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=C6347BDEEF1D47C4AA79FA7BDEB8754E, vertexId=vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954]
2016-05-17 21:30:02,069 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=C6347BDEEF1D47C4AA79FA7BDEB8754E, vertexName=D5D222F5273044BD9FB8043377CC2954, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:02,069 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=C6347BDEEF1D47C4AA79FA7BDEB8754E, vertexId=vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954]
2016-05-17 21:30:02,070 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:02,070 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601840_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D5D222F5273044BD9FB8043377CC2954, vertexId=vertex_1463520601840_0001_1_00, initRequestedTime=1463520602054, initedTime=1463520602070, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:02,071 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954] with distanceFromRoot: 0
2016-05-17 21:30:02,071 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:02,071 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D5D222F5273044BD9FB8043377CC2954
2016-05-17 21:30:02,071 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954]
2016-05-17 21:30:02,071 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:02,071 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601840_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520601840_0001_1_00, startRequestedTime=1463520602071, startedTime=1463520602071
2016-05-17 21:30:02,072 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:02,072 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:02,072 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601840_0001_1][Event:TASK_STARTED]: vertexName=D5D222F5273044BD9FB8043377CC2954, taskId=task_1463520601840_0001_1_00_000000, scheduledTime=1463520602072, launchTime=1463520602072
2016-05-17 21:30:02,073 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520601840_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:02,073 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954]
2016-05-17 21:30:02,073 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [75C99928FD4F4A40818F88BA1E5CCADF/C7A8D603E5434A538BFF1465F382B824] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS, VertexName: D5D222F5273044BD9FB8043377CC2954, VertexParallelism: 1, TaskAttemptID:attempt_1463520601840_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=43AB75A595214468826EDE0D9CEF2476, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C6347BDEEF1D47C4AA79FA7BDEB8754E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:02,073 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520601840_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:02,074 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:02,075 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:02,075 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:02,075 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:02,076 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520601840_0001_00_000001
2016-05-17 21:30:02,076 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520601840_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:02,078 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520601840_0001_00_000001
2016-05-17 21:30:02,078 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:02,079 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:02,080 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520601840_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:02,080 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601840_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520601840_0001_00_000001, launchTime=1463520602079
2016-05-17 21:30:02,082 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520601840_0001_00_000001
2016-05-17 21:30:02,083 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520601840_0001_1_00_000000_0] to container: [container_1463520601840_0001_00_000001]
2016-05-17 21:30:02,083 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520601840_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:02,085 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520601840_0001_1_00_000000_0] started. Is using containerId: [container_1463520601840_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:02,085 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520601840_0001_00_000001 given task: attempt_1463520601840_0001_1_00_000000_0
2016-05-17 21:30:02,085 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520601840_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520601840_0001_1_00_000000_0
2016-05-17 21:30:02,085 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601840_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D5D222F5273044BD9FB8043377CC2954, taskAttemptId=attempt_1463520601840_0001_1_00_000000_0, startTime=1463520602084, containerId=container_1463520601840_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520601840_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:02,085 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:02,086 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520601840_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:02,086 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520601840_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:02 Starting to run new task attempt: attempt_1463520601840_0001_1_00_000000_0
2016-05-17 21:30:02,088 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520601840_0001_1_00_000000_0
2016-05-17 21:30:02,089 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:02,089 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:02,090 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:02,090 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:02,090 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:02,090 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520601840_0001, with dag id: dag_1463520601840_0001_1
2016-05-17 21:30:02,091 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:02,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [75C99928FD4F4A40818F88BA1E5CCADF/C7A8D603E5434A538BFF1465F382B824] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS, VertexName: D5D222F5273044BD9FB8043377CC2954, VertexParallelism: 1, TaskAttemptID:attempt_1463520601840_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=43AB75A595214468826EDE0D9CEF2476, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C6347BDEEF1D47C4AA79FA7BDEB8754E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:02,093 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:02,096 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520601840_0001_1_00_000000_0
2016-05-17 21:30:02,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:02,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:02,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=43AB75A595214468826EDE0D9CEF2476, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:02,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:02,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 43AB75A595214468826EDE0D9CEF2476
2016-05-17 21:30:02,099 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 43AB75A595214468826EDE0D9CEF2476
2016-05-17 21:30:02,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:02,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C6347BDEEF1D47C4AA79FA7BDEB8754E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:02,099 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520601840_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:02,101 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:02,102 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C6347BDEEF1D47C4AA79FA7BDEB8754E
2016-05-17 21:30:02,102 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:02,103 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: C6347BDEEF1D47C4AA79FA7BDEB8754E
2016-05-17 21:30:02,106 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:02,107 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:02,107 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 43AB75A595214468826EDE0D9CEF2476
2016-05-17 21:30:02,107 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 43AB75A595214468826EDE0D9CEF2476
2016-05-17 21:30:02,121 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:02,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C6347BDEEF1D47C4AA79FA7BDEB8754E
2016-05-17 21:30:02,121 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D5D222F5273044BD9FB8043377CC2954, ordinal: 0
2016-05-17 21:30:02,122 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5D222F5273044BD9FB8043377CC2954, mem on start (mb), free: 145, total: 397, max: 672
2016-05-17 21:30:02,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:02,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:02,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:02,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:02,123 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:02,123 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:02,123 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:02,123 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:02,123 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:02,124 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:02,124 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:02,124 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 43AB75A595214468826EDE0D9CEF2476, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:02,124 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C6347BDEEF1D47C4AA79FA7BDEB8754E, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:02,125 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:02,125 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 43AB75A595214468826EDE0D9CEF2476
2016-05-17 21:30:02,125 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:02,125 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 43AB75A595214468826EDE0D9CEF2476
2016-05-17 21:30:02,125 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 43AB75A595214468826EDE0D9CEF2476 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:02,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:02,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:02,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:02,127 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520601840_0001_1_00_000000_0
2016-05-17 21:30:02,127 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:02,128 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@70265461
2016-05-17 21:30:02,129 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@70265461
2016-05-17 21:30:02,129 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:02,129 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:02,132 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:02,132 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 43AB75A595214468826EDE0D9CEF2476
2016-05-17 21:30:02,133 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS"], id: C6347BDEEF1D47C4AA79FA7BDEB8754E
2016-05-17 21:30:02,133 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS"]
2016-05-17 21:30:02,133 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:02,133 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:02,134 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D5D222F5273044BD9FB8043377CC2954, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:02,136 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:02,137 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:02,138 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520601840_0001_1_00_000000_0
2016-05-17 21:30:02,138 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520601840_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:02,139 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520601840_0001_1_00_000000_0
2016-05-17 21:30:02,139 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5D222F5273044BD9FB8043377CC2954, mem on close (mb), free: 143, total: 397, max: 672
2016-05-17 21:30:02 Completed running task attempt: attempt_1463520601840_0001_1_00_000000_0
2016-05-17 21:30:02,140 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520601840_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:02,141 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520602134, Process_Duration=5, Process_End_Time=1463520602139, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:02,141 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:02,142 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:02,143 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:02,142 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601840_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D5D222F5273044BD9FB8043377CC2954, taskAttemptId=attempt_1463520601840_0001_1_00_000000_0, startTime=1463520602084, finishTime=1463520602141, timeTaken=57, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520602134, Process_Duration=5, Process_End_Time=1463520602139, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:02,143 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520601840_0001_00_000001
2016-05-17 21:30:02,144 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520601840_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:02,145 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:02,145 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520601840_0001_1_00_000000_0
2016-05-17 21:30:02,146 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:02,146 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601840_0001_1][Event:TASK_FINISHED]: vertexName=D5D222F5273044BD9FB8043377CC2954, taskId=task_1463520601840_0001_1_00_000000, startTime=1463520602084, finishTime=1463520602145, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1463520601840_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520602134, Process_Duration=5, Process_End_Time=1463520602139, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:02,146 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520601840_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:02,147 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954] : 1
2016-05-17 21:30:02,147 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954]
2016-05-17 21:30:02,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:02,149 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601840_0001_1][Event:VERTEX_FINISHED]: vertexName=D5D222F5273044BD9FB8043377CC2954, vertexId=vertex_1463520601840_0001_1_00, initRequestedTime=1463520602054, initedTime=1463520602070, startRequestedTime=1463520602071, startedTime=1463520602071, finishTime=1463520602147, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520602134, Process_Duration=5, Process_End_Time=1463520602139, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520602084, firstTasksToStart=[ task_1463520601840_0001_1_00_000000 ], lastTaskFinishTime=1463520602141, lastTasksToFinish=[ task_1463520601840_0001_1_00_000000 ], minTaskDuration=57, maxTaskDuration=57, avgTaskDuration=57.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520601840_0001_1_00_000000 ], longestDurationTasks=[ task_1463520601840_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:02,149 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:02,149 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520601840_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:02,150 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601840_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520601840_0001_00_000001, stoppedTime=1463520602150, exitStatus=0
2016-05-17 21:30:02,150 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520601840_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:02,150 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:02,151 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:02,178 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520601840_0001_1
2016-05-17 21:30:02,178 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601840_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520601840_0001_1
2016-05-17 21:30:02,178 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: C6347BDEEF1D47C4AA79FA7BDEB8754E for vertex: vertex_1463520601840_0001_1_00 [D5D222F5273044BD9FB8043377CC2954]
2016-05-17 21:30:02,178 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520601840_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:02,179 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520601840_0001_00_000001: 34 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:02,179 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520601840_0001_00_000001, Exiting
2016-05-17 21:30:02,179 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520601840_0001_00_000001
2016-05-17 21:30:02,179 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520601840_0001_00_000001
2016-05-17 21:30:02,183 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520601840_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:02,184 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520601840_0001_00_000001 completed successfully
2016-05-17 21:30:02,195 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520601840_0001_1, queueSize=0
2016-05-17 21:30:02,195 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:02,195 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520601840_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520601840_0001_1, startTime=1463520602053, finishTime=1463520602193, timeTaken=140, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520602134, Process_Duration=5, Process_End_Time=1463520602139, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:02,195 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520601840_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:02,197 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520601840_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:02,197 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520601840_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:02,197 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520601840_0001_00_000001
2016-05-17 21:30:02 Completed Dag: dag_1463520601840_0001_1
2016-05-17 21:30:02,199 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520601840_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:02,199 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520601840_0001_1_post
2016-05-17 21:30:02,199 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:02,200 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:02,205 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...SULTS.-1_'field2'_RESULTS, applicationId=application_1463520601840_0001
2016-05-17 21:30:02,208 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:02,208 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:02,208 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:02,208 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:02,208 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:02,209 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:02,209 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:02,210 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:02,211 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:02,211 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:02,211 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:02,212 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:02,212 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:02,218 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/75C99928FD4F4A40818F88BA1E5CCADF/C7A8D603E5434A538BFF1465F382B824/.tez/application_1463520601840_0001
2016-05-17 21:30:02,218 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS/_temporary
2016-05-17 21:30:02,219 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.409
2016-05-17 21:30:02,298 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_RESULTS
2016-05-17 21:30:02,322 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:02,322 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:02,377 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-17 21:30:02,385 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.031
2016-05-17 21:30:02,386 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:02,386 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:02,386 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:02,403 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:02,403 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL
2016-05-17 21:30:02,415 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:02,416 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:02,416 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL"]
2016-05-17 21:30:02,416 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:02,416 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:02,416 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:02,417 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-17 21:30:02,417 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:02,431 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:02,436 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:02,437 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:02,437 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:02,447 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8FB64E97FAED4AA6963FC7A7661FB8E8/A63C4423C216482387A393AE4EF055CE/.tez/application_1463520602436_0001 doesn't exist and is created
2016-05-17 21:30:02,463 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:02,463 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:02,464 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8FB64E97FAED4AA6963FC7A7661FB8E8/A63C4423C216482387A393AE4EF055CE/.tez/application_1463520602436_0001
2016-05-17 21:30:02,467 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520602436_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:02,481 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:02,482 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:02,484 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:02,484 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:02,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8FB64E97FAED4AA6963FC7A7661FB8E8/A63C4423C216482387A393AE4EF055CE/.tez/application_1463520602436_0001/localmode-log-dir
2016-05-17 21:30:02,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8FB64E97FAED4AA6963FC7A7661FB8E8/A63C4423C216482387A393AE4EF055CE/.tez/application_1463520602436_0001/localmode-log-dir/history.txt.appattempt_1463520602436_0001_000000, maxErrors=10
2016-05-17 21:30:02,485 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:02,485 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520602436_0001_000000, appSubmitTime=1463520602467, launchTime=1463520602467
2016-05-17 21:30:02,490 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:02,491 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:02,492 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:02,492 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:02,493 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:46439
2016-05-17 21:30:02,496 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:02,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520602436_0001_000000, startTime=1463520602498
2016-05-17 21:30:02,498 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:02,499 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:02,563 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:02,563 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:02,564 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_'field2'_ALL, applicationId=application_1463520602436_0001, dagName=[8FB64E97FAED4AA6963FC7A7661FB8E8/A63C4423C216482387A393AE4EF055CE] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-17 21:30:02,586 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8FB64E97FAED4AA6963FC7A7661FB8E8/A63C4423C216482387A393AE4EF055CE] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-17 21:30:02,589 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520602436_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8FB64E97FAED4AA6963FC7A7661FB8E8/A63C4423C216482387A393AE4EF055CE/.tez/application_1463520602436_0001/localmode-log-dir/dag_1463520602436_0001_1.dot
2016-05-17 21:30:02,591 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8FB64E97FAED4AA6963FC7A7661FB8E8/A63C4423C216482387A393AE4EF055CE/.tez/application_1463520602436_0001/localmode-log-dir/dag_1463520602436_0001_1-tez-dag.pb.txt
2016-05-17 21:30:02,619 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520602436_0001_1
2016-05-17 21:30:02,619 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:02,619 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8FB64E97FAED4AA6963FC7A7661FB8E8/A63C4423C216482387A393AE4EF055CE] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-17 21:30:02 Running Dag: dag_1463520602436_0001_1
2016-05-17 21:30:02,636 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602436_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520602436_0001_1, submitTime=1463520602587
2016-05-17 21:30:02,636 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:02,637 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BDB4DF1B4A244679A0B36F5C1DE56ECE
2016-05-17 21:30:02,637 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:02,638 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602436_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520602436_0001_1, initTime=1463520602636
2016-05-17 21:30:02,638 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:02,638 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520602436_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:02,639 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602436_0001_1][Event:DAG_STARTED]: dagID=dag_1463520602436_0001_1, startTime=1463520602639
2016-05-17 21:30:02,639 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:02,639 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:02,640 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520602436_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:02,641 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BDB4DF1B4A244679A0B36F5C1DE56ECE : {52F271F447D14645A8B72117D7B1D5B9={InputName=52F271F447D14645A8B72117D7B1D5B9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:02,641 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 52F271F447D14645A8B72117D7B1D5B9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:02,641 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE]
2016-05-17 21:30:02,641 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE]
2016-05-17 21:30:02,641 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE]
2016-05-17 21:30:02,641 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE]. Starting root input initializers: 1
2016-05-17 21:30:02,642 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:02,642 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 52F271F447D14645A8B72117D7B1D5B9 on vertex vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE]
2016-05-17 21:30:02,642 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_'field2'_ALL, applicationId=application_1463520602436_0001, dagName=[8FB64E97FAED4AA6963FC7A7661FB8E8/A63C4423C216482387A393AE4EF055CE] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-17 21:30:02,644 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:02,645 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 52F271F447D14645A8B72117D7B1D5B9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:02,645 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:02,645 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:02,648 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:02,648 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 52F271F447D14645A8B72117D7B1D5B9 on vertex vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE]
2016-05-17 21:30:02,649 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:02,649 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {52F271F447D14645A8B72117D7B1D5B9=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:02,649 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE] parallelism set to 1
2016-05-17 21:30:02,649 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:02,650 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602436_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520602436_0001_1_00, eventCount=1
2016-05-17 21:30:02,650 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE]
2016-05-17 21:30:02,650 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D9E9F6926BF04D2992E47B150D6C3CCE, vertexId=vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:02,650 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D9E9F6926BF04D2992E47B150D6C3CCE, vertexId=vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE]
2016-05-17 21:30:02,653 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D9E9F6926BF04D2992E47B150D6C3CCE, vertexName=BDB4DF1B4A244679A0B36F5C1DE56ECE, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:02,653 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D9E9F6926BF04D2992E47B150D6C3CCE, vertexId=vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE]
2016-05-17 21:30:02,654 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:02,654 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602436_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BDB4DF1B4A244679A0B36F5C1DE56ECE, vertexId=vertex_1463520602436_0001_1_00, initRequestedTime=1463520602641, initedTime=1463520602654, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:02,655 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE] with distanceFromRoot: 0
2016-05-17 21:30:02,655 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:02,655 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BDB4DF1B4A244679A0B36F5C1DE56ECE
2016-05-17 21:30:02,655 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE]
2016-05-17 21:30:02,656 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:02,656 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602436_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520602436_0001_1_00, startRequestedTime=1463520602655, startedTime=1463520602655
2016-05-17 21:30:02,656 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:02,657 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:02,657 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602436_0001_1][Event:TASK_STARTED]: vertexName=BDB4DF1B4A244679A0B36F5C1DE56ECE, taskId=task_1463520602436_0001_1_00_000000, scheduledTime=1463520602656, launchTime=1463520602656
2016-05-17 21:30:02,657 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520602436_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:02,657 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE]
2016-05-17 21:30:02,657 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8FB64E97FAED4AA6963FC7A7661FB8E8/A63C4423C216482387A393AE4EF055CE] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL, VertexName: BDB4DF1B4A244679A0B36F5C1DE56ECE, VertexParallelism: 1, TaskAttemptID:attempt_1463520602436_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=52F271F447D14645A8B72117D7B1D5B9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D9E9F6926BF04D2992E47B150D6C3CCE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:02,658 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520602436_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:02,658 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:02,660 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:02,661 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:02,661 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:02,662 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520602436_0001_00_000001
2016-05-17 21:30:02,662 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520602436_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:02,665 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520602436_0001_00_000001
2016-05-17 21:30:02,665 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:02,666 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520602436_0001, with dag id: dag_1463520602436_0001_1
2016-05-17 21:30:02,666 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:02,667 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520602436_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:02,667 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602436_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520602436_0001_00_000001, launchTime=1463520602666
2016-05-17 21:30:02,669 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520602436_0001_00_000001
2016-05-17 21:30:02,670 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520602436_0001_1_00_000000_0] to container: [container_1463520602436_0001_00_000001]
2016-05-17 21:30:02,670 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520602436_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:02,670 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520602436_0001_00_000001 given task: attempt_1463520602436_0001_1_00_000000_0
2016-05-17 21:30:02,671 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520602436_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520602436_0001_1_00_000000_0
2016-05-17 21:30:02,670 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520602436_0001_1_00_000000_0] started. Is using containerId: [container_1463520602436_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:02,672 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:02,672 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602436_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BDB4DF1B4A244679A0B36F5C1DE56ECE, taskAttemptId=attempt_1463520602436_0001_1_00_000000_0, startTime=1463520602670, containerId=container_1463520602436_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520602436_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:02,672 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520602436_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:02 Starting to run new task attempt: attempt_1463520602436_0001_1_00_000000_0
2016-05-17 21:30:02,673 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520602436_0001_1_00_000000_0
2016-05-17 21:30:02,673 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:02,673 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:02,673 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:02,673 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:02,673 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:02,674 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:02,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8FB64E97FAED4AA6963FC7A7661FB8E8/A63C4423C216482387A393AE4EF055CE] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL, VertexName: BDB4DF1B4A244679A0B36F5C1DE56ECE, VertexParallelism: 1, TaskAttemptID:attempt_1463520602436_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=52F271F447D14645A8B72117D7B1D5B9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D9E9F6926BF04D2992E47B150D6C3CCE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:02,673 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520602436_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:02,674 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:02,675 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520602436_0001_1_00_000000_0
2016-05-17 21:30:02,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:02,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:02,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:02,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D9E9F6926BF04D2992E47B150D6C3CCE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:02,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:02,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D9E9F6926BF04D2992E47B150D6C3CCE
2016-05-17 21:30:02,677 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:02,677 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D9E9F6926BF04D2992E47B150D6C3CCE
2016-05-17 21:30:02,678 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:02,681 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520602436_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:02,681 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=52F271F447D14645A8B72117D7B1D5B9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:02,682 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:02,682 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 52F271F447D14645A8B72117D7B1D5B9
2016-05-17 21:30:02,682 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 52F271F447D14645A8B72117D7B1D5B9
2016-05-17 21:30:02,688 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:02,688 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 52F271F447D14645A8B72117D7B1D5B9
2016-05-17 21:30:02,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 52F271F447D14645A8B72117D7B1D5B9
2016-05-17 21:30:02,692 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BDB4DF1B4A244679A0B36F5C1DE56ECE, ordinal: 0
2016-05-17 21:30:02,693 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDB4DF1B4A244679A0B36F5C1DE56ECE, mem on start (mb), free: 285, total: 397, max: 672
2016-05-17 21:30:02,693 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:02,693 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:02,693 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:02,694 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:02,694 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D9E9F6926BF04D2992E47B150D6C3CCE
2016-05-17 21:30:02,694 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:02,694 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:02,695 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:02,695 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:02,695 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:02,695 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:02,696 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:02,696 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:02,696 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D9E9F6926BF04D2992E47B150D6C3CCE, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:02,696 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 52F271F447D14645A8B72117D7B1D5B9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:02,696 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:02,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 52F271F447D14645A8B72117D7B1D5B9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:02,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 52F271F447D14645A8B72117D7B1D5B9
2016-05-17 21:30:02,697 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:02,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 52F271F447D14645A8B72117D7B1D5B9
2016-05-17 21:30:02,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:02,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:02,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:02,698 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520602436_0001_1_00_000000_0
2016-05-17 21:30:02,699 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:02,700 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@53289638
2016-05-17 21:30:02,701 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@53289638
2016-05-17 21:30:02,701 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:02,702 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:02,702 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:02,703 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 52F271F447D14645A8B72117D7B1D5B9
2016-05-17 21:30:02,704 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL"], id: D9E9F6926BF04D2992E47B150D6C3CCE
2016-05-17 21:30:02,704 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL"]
2016-05-17 21:30:02,704 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:02,704 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:02,705 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BDB4DF1B4A244679A0B36F5C1DE56ECE, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:02,708 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:02,708 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:02,708 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520602436_0001_1_00_000000_0
2016-05-17 21:30:02,709 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520602436_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:02,710 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520602436_0001_1_00_000000_0
2016-05-17 21:30:02,710 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDB4DF1B4A244679A0B36F5C1DE56ECE, mem on close (mb), free: 283, total: 397, max: 672
2016-05-17 21:30:02 Completed running task attempt: attempt_1463520602436_0001_1_00_000000_0
2016-05-17 21:30:02,711 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520602436_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:02,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520602705, Process_Duration=5, Process_End_Time=1463520602710, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:02,712 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:02,713 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:02,713 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:02,713 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520602436_0001_00_000001
2016-05-17 21:30:02,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602436_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BDB4DF1B4A244679A0B36F5C1DE56ECE, taskAttemptId=attempt_1463520602436_0001_1_00_000000_0, startTime=1463520602670, finishTime=1463520602712, timeTaken=42, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520602705, Process_Duration=5, Process_End_Time=1463520602710, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:02,715 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520602436_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:02,715 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:02,715 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520602436_0001_1_00_000000_0
2016-05-17 21:30:02,716 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602436_0001_1][Event:TASK_FINISHED]: vertexName=BDB4DF1B4A244679A0B36F5C1DE56ECE, taskId=task_1463520602436_0001_1_00_000000, startTime=1463520602670, finishTime=1463520602716, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1463520602436_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520602705, Process_Duration=5, Process_End_Time=1463520602710, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:02,717 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520602436_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:02,717 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE] : 1
2016-05-17 21:30:02,717 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE]
2016-05-17 21:30:02,717 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:02,718 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:02,719 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602436_0001_1][Event:VERTEX_FINISHED]: vertexName=BDB4DF1B4A244679A0B36F5C1DE56ECE, vertexId=vertex_1463520602436_0001_1_00, initRequestedTime=1463520602641, initedTime=1463520602654, startRequestedTime=1463520602655, startedTime=1463520602655, finishTime=1463520602717, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520602705, Process_Duration=5, Process_End_Time=1463520602710, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520602670, firstTasksToStart=[ task_1463520602436_0001_1_00_000000 ], lastTaskFinishTime=1463520602712, lastTasksToFinish=[ task_1463520602436_0001_1_00_000000 ], minTaskDuration=42, maxTaskDuration=42, avgTaskDuration=42.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520602436_0001_1_00_000000 ], longestDurationTasks=[ task_1463520602436_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:02,719 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:02,719 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520602436_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:02,720 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602436_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520602436_0001_00_000001, stoppedTime=1463520602720, exitStatus=0
2016-05-17 21:30:02,720 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:02,720 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520602436_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:02,720 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:02,720 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520602436_0001_1
2016-05-17 21:30:02,721 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602436_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520602436_0001_1
2016-05-17 21:30:02,721 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D9E9F6926BF04D2992E47B150D6C3CCE for vertex: vertex_1463520602436_0001_1_00 [BDB4DF1B4A244679A0B36F5C1DE56ECE]
2016-05-17 21:30:02,725 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520602436_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:02,725 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520602436_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:02,725 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520602436_0001_00_000001, Exiting
2016-05-17 21:30:02,725 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520602436_0001_00_000001
2016-05-17 21:30:02,726 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520602436_0001_00_000001
2016-05-17 21:30:02,728 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520602436_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:02,728 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520602436_0001_00_000001 completed successfully
2016-05-17 21:30:02,732 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520602436_0001_1, queueSize=0
2016-05-17 21:30:02,733 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:02,733 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602436_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520602436_0001_1, startTime=1463520602639, finishTime=1463520602731, timeTaken=92, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520602705, Process_Duration=5, Process_End_Time=1463520602710, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:02,733 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520602436_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:02,733 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520602436_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:02,733 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520602436_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:02 Completed Dag: dag_1463520602436_0001_1
2016-05-17 21:30:02,734 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520602436_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:02,734 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520602436_0001_1_post
2016-05-17 21:30:02,734 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:02,734 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:02,735 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520602436_0001_00_000001
2016-05-17 21:30:02,742 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...'_RESULTS.-1_'field2'_ALL, applicationId=application_1463520602436_0001
2016-05-17 21:30:02,744 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:02,744 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:02,744 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:02,744 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:02,744 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:02,745 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:02,746 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:02,746 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:02,746 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:02,746 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:02,747 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:02,748 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:02,749 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:02,753 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8FB64E97FAED4AA6963FC7A7661FB8E8/A63C4423C216482387A393AE4EF055CE/.tez/application_1463520602436_0001
2016-05-17 21:30:02,753 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL/_temporary
2016-05-17 21:30:02,754 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.338
2016-05-17 21:30:02,832 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_ALL
2016-05-17 21:30:02,857 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:02,858 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:02,905 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-17 21:30:02,919 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.031
2016-05-17 21:30:02,920 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:02,920 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:02,920 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:02,937 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:02,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1
2016-05-17 21:30:02,953 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:02,953 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:02,954 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1"]
2016-05-17 21:30:02,954 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:02,955 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:02,955 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:02,956 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-17 21:30:02,956 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:02,971 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:02,976 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:02,976 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:02,977 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:02,986 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FBFBE1E9423F4AC78C47B38BEEA929C2/B272B4897B464A2D84E66B630B5F0035/.tez/application_1463520602976_0001 doesn't exist and is created
2016-05-17 21:30:03,003 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:03,003 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:03,004 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FBFBE1E9423F4AC78C47B38BEEA929C2/B272B4897B464A2D84E66B630B5F0035/.tez/application_1463520602976_0001
2016-05-17 21:30:03,007 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520602976_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:03,020 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:03,020 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:03,022 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:03,023 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:03,023 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FBFBE1E9423F4AC78C47B38BEEA929C2/B272B4897B464A2D84E66B630B5F0035/.tez/application_1463520602976_0001/localmode-log-dir
2016-05-17 21:30:03,023 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FBFBE1E9423F4AC78C47B38BEEA929C2/B272B4897B464A2D84E66B630B5F0035/.tez/application_1463520602976_0001/localmode-log-dir/history.txt.appattempt_1463520602976_0001_000000, maxErrors=10
2016-05-17 21:30:03,023 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:03,023 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520602976_0001_000000, appSubmitTime=1463520603007, launchTime=1463520603007
2016-05-17 21:30:03,030 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:50165
2016-05-17 21:30:03,031 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:03,033 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:03,034 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:03,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:03,034 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:03,037 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520602976_0001_000000, startTime=1463520603037
2016-05-17 21:30:03,037 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:03,037 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:03,103 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:03,103 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:03,104 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, applicationId=application_1463520602976_0001, dagName=[FBFBE1E9423F4AC78C47B38BEEA929C2/B272B4897B464A2D84E66B630B5F0035] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-17 21:30:03,123 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [FBFBE1E9423F4AC78C47B38BEEA929C2/B272B4897B464A2D84E66B630B5F0035] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-17 21:30:03,126 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520602976_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FBFBE1E9423F4AC78C47B38BEEA929C2/B272B4897B464A2D84E66B630B5F0035/.tez/application_1463520602976_0001/localmode-log-dir/dag_1463520602976_0001_1.dot
2016-05-17 21:30:03,127 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FBFBE1E9423F4AC78C47B38BEEA929C2/B272B4897B464A2D84E66B630B5F0035/.tez/application_1463520602976_0001/localmode-log-dir/dag_1463520602976_0001_1-tez-dag.pb.txt
2016-05-17 21:30:03,154 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520602976_0001_1
2016-05-17 21:30:03,154 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:03,155 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [FBFBE1E9423F4AC78C47B38BEEA929C2/B272B4897B464A2D84E66B630B5F0035] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-17 21:30:03 Running Dag: dag_1463520602976_0001_1
2016-05-17 21:30:03,172 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602976_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520602976_0001_1, submitTime=1463520603124
2016-05-17 21:30:03,172 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:03,173 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CA0AB2B28D7D416CA0638356633496E7
2016-05-17 21:30:03,174 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:03,175 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:03,175 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602976_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520602976_0001_1, initTime=1463520603173
2016-05-17 21:30:03,176 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520602976_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:03,177 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602976_0001_1][Event:DAG_STARTED]: dagID=dag_1463520602976_0001_1, startTime=1463520603176
2016-05-17 21:30:03,177 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:03,178 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520602976_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:03,178 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CA0AB2B28D7D416CA0638356633496E7 : {70B5785EDA724FBB8BBD56228EC8445D={InputName=70B5785EDA724FBB8BBD56228EC8445D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:03,178 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 70B5785EDA724FBB8BBD56228EC8445D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:03,178 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7]
2016-05-17 21:30:03,178 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7]
2016-05-17 21:30:03,178 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7]
2016-05-17 21:30:03,177 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:03,179 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7]. Starting root input initializers: 1
2016-05-17 21:30:03,179 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:03,179 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, applicationId=application_1463520602976_0001, dagName=[FBFBE1E9423F4AC78C47B38BEEA929C2/B272B4897B464A2D84E66B630B5F0035] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-17 21:30:03,180 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 70B5785EDA724FBB8BBD56228EC8445D on vertex vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7]
2016-05-17 21:30:03,183 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:03,183 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 70B5785EDA724FBB8BBD56228EC8445D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:03,184 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:03,184 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:03,187 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:03,191 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 70B5785EDA724FBB8BBD56228EC8445D on vertex vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7]
2016-05-17 21:30:03,191 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:03,191 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {70B5785EDA724FBB8BBD56228EC8445D=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:03,192 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7] parallelism set to 1
2016-05-17 21:30:03,192 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:03,192 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602976_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520602976_0001_1_00, eventCount=1
2016-05-17 21:30:03,192 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7]
2016-05-17 21:30:03,192 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9E8346D42036427CBB08CC2AD1E3F428, vertexId=vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:03,193 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9E8346D42036427CBB08CC2AD1E3F428, vertexId=vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7]
2016-05-17 21:30:03,196 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9E8346D42036427CBB08CC2AD1E3F428, vertexName=CA0AB2B28D7D416CA0638356633496E7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:03,196 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9E8346D42036427CBB08CC2AD1E3F428, vertexId=vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7]
2016-05-17 21:30:03,198 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:03,198 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602976_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CA0AB2B28D7D416CA0638356633496E7, vertexId=vertex_1463520602976_0001_1_00, initRequestedTime=1463520603178, initedTime=1463520603198, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:03,199 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7] with distanceFromRoot: 0
2016-05-17 21:30:03,199 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:03,199 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CA0AB2B28D7D416CA0638356633496E7
2016-05-17 21:30:03,200 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7]
2016-05-17 21:30:03,201 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:03,201 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602976_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520602976_0001_1_00, startRequestedTime=1463520603199, startedTime=1463520603199
2016-05-17 21:30:03,202 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:03,203 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:03,203 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602976_0001_1][Event:TASK_STARTED]: vertexName=CA0AB2B28D7D416CA0638356633496E7, taskId=task_1463520602976_0001_1_00_000000, scheduledTime=1463520603202, launchTime=1463520603202
2016-05-17 21:30:03,204 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520602976_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:03,204 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7]
2016-05-17 21:30:03,208 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FBFBE1E9423F4AC78C47B38BEEA929C2/B272B4897B464A2D84E66B630B5F0035] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, VertexName: CA0AB2B28D7D416CA0638356633496E7, VertexParallelism: 1, TaskAttemptID:attempt_1463520602976_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=70B5785EDA724FBB8BBD56228EC8445D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9E8346D42036427CBB08CC2AD1E3F428, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:03,209 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520602976_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:03,210 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:03,212 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:03,213 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:03,214 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:03,215 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520602976_0001_00_000001
2016-05-17 21:30:03,215 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520602976_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:03,218 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520602976_0001_00_000001
2016-05-17 21:30:03,218 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:03,219 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520602976_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:03,219 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:03,220 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602976_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520602976_0001_00_000001, launchTime=1463520603219
2016-05-17 21:30:03,220 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520602976_0001_00_000001
2016-05-17 21:30:03,222 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520602976_0001_1_00_000000_0] to container: [container_1463520602976_0001_00_000001]
2016-05-17 21:30:03,223 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520602976_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:03,223 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520602976_0001, with dag id: dag_1463520602976_0001_1
2016-05-17 21:30:03,224 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520602976_0001_00_000001 given task: attempt_1463520602976_0001_1_00_000000_0
2016-05-17 21:30:03,225 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520602976_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520602976_0001_1_00_000000_0
2016-05-17 21:30:03,224 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520602976_0001_1_00_000000_0] started. Is using containerId: [container_1463520602976_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:03,226 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602976_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CA0AB2B28D7D416CA0638356633496E7, taskAttemptId=attempt_1463520602976_0001_1_00_000000_0, startTime=1463520603224, containerId=container_1463520602976_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520602976_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:03,226 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520602976_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:03,226 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:03,228 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520602976_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:03 Starting to run new task attempt: attempt_1463520602976_0001_1_00_000000_0
2016-05-17 21:30:03,229 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520602976_0001_1_00_000000_0
2016-05-17 21:30:03,229 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:03,229 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:03,229 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:03,230 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:03,230 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:03,230 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:03,230 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FBFBE1E9423F4AC78C47B38BEEA929C2/B272B4897B464A2D84E66B630B5F0035] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, VertexName: CA0AB2B28D7D416CA0638356633496E7, VertexParallelism: 1, TaskAttemptID:attempt_1463520602976_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=70B5785EDA724FBB8BBD56228EC8445D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9E8346D42036427CBB08CC2AD1E3F428, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:03,230 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:03,232 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520602976_0001_1_00_000000_0
2016-05-17 21:30:03,235 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:03,235 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:03,234 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520602976_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:03,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=70B5785EDA724FBB8BBD56228EC8445D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:03,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:03,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 70B5785EDA724FBB8BBD56228EC8445D
2016-05-17 21:30:03,240 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 70B5785EDA724FBB8BBD56228EC8445D
2016-05-17 21:30:03,237 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9E8346D42036427CBB08CC2AD1E3F428, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:03,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:03,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:03,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9E8346D42036427CBB08CC2AD1E3F428
2016-05-17 21:30:03,243 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:03,243 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9E8346D42036427CBB08CC2AD1E3F428
2016-05-17 21:30:03,245 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:03,257 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:03,257 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 70B5785EDA724FBB8BBD56228EC8445D
2016-05-17 21:30:03,257 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 70B5785EDA724FBB8BBD56228EC8445D
2016-05-17 21:30:03,263 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CA0AB2B28D7D416CA0638356633496E7, ordinal: 0
2016-05-17 21:30:03,264 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA0AB2B28D7D416CA0638356633496E7, mem on start (mb), free: 208, total: 397, max: 672
2016-05-17 21:30:03,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:03,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:03,267 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:03,269 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:03,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9E8346D42036427CBB08CC2AD1E3F428
2016-05-17 21:30:03,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:03,270 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:03,270 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:03,270 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:03,271 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:03,271 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:03,271 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:03,271 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:03,272 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 70B5785EDA724FBB8BBD56228EC8445D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:03,272 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9E8346D42036427CBB08CC2AD1E3F428, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:03,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:03,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 70B5785EDA724FBB8BBD56228EC8445D
2016-05-17 21:30:03,273 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:03,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 70B5785EDA724FBB8BBD56228EC8445D
2016-05-17 21:30:03,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 70B5785EDA724FBB8BBD56228EC8445D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:03,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:03,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:03,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:03,275 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520602976_0001_1_00_000000_0
2016-05-17 21:30:03,277 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:03,278 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@8cb83c4
2016-05-17 21:30:03,279 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:03,280 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 70B5785EDA724FBB8BBD56228EC8445D
2016-05-17 21:30:03,280 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1"], id: 9E8346D42036427CBB08CC2AD1E3F428
2016-05-17 21:30:03,280 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@8cb83c4
2016-05-17 21:30:03,282 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:03,281 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1"]
2016-05-17 21:30:03,283 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:03,284 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:03,284 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:03,285 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CA0AB2B28D7D416CA0638356633496E7, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:30:03,289 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:03,289 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:03,290 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520602976_0001_1_00_000000_0
2016-05-17 21:30:03,290 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520602976_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:03,292 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520602976_0001_1_00_000000_0
2016-05-17 21:30:03,292 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA0AB2B28D7D416CA0638356633496E7, mem on close (mb), free: 206, total: 397, max: 672
2016-05-17 21:30:03 Completed running task attempt: attempt_1463520602976_0001_1_00_000000_0
2016-05-17 21:30:03,294 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520602976_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:03,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520603285, Process_Duration=6, Process_End_Time=1463520603291, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:03,297 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:03,297 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602976_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CA0AB2B28D7D416CA0638356633496E7, taskAttemptId=attempt_1463520602976_0001_1_00_000000_0, startTime=1463520603224, finishTime=1463520603297, timeTaken=73, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520603285, Process_Duration=6, Process_End_Time=1463520603291, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:03,298 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:03,299 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520602976_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:03,299 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:03,299 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520602976_0001_1_00_000000_0
2016-05-17 21:30:03,299 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:03,300 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:03,300 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602976_0001_1][Event:TASK_FINISHED]: vertexName=CA0AB2B28D7D416CA0638356633496E7, taskId=task_1463520602976_0001_1_00_000000, startTime=1463520603224, finishTime=1463520603299, timeTaken=75, status=SUCCEEDED, successfulAttemptID=attempt_1463520602976_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520603285, Process_Duration=6, Process_End_Time=1463520603291, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:03,300 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520602976_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:03,301 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7] : 1
2016-05-17 21:30:03,301 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7]
2016-05-17 21:30:03,299 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520602976_0001_00_000001
2016-05-17 21:30:03,304 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:03,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602976_0001_1][Event:VERTEX_FINISHED]: vertexName=CA0AB2B28D7D416CA0638356633496E7, vertexId=vertex_1463520602976_0001_1_00, initRequestedTime=1463520603178, initedTime=1463520603198, startRequestedTime=1463520603199, startedTime=1463520603199, finishTime=1463520603302, timeTaken=103, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520603285, Process_Duration=6, Process_End_Time=1463520603291, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520603224, firstTasksToStart=[ task_1463520602976_0001_1_00_000000 ], lastTaskFinishTime=1463520603297, lastTasksToFinish=[ task_1463520602976_0001_1_00_000000 ], minTaskDuration=73, maxTaskDuration=73, avgTaskDuration=73.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520602976_0001_1_00_000000 ], longestDurationTasks=[ task_1463520602976_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:03,305 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:03,306 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520602976_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:03,306 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:03,306 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602976_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520602976_0001_00_000001, stoppedTime=1463520603306, exitStatus=0
2016-05-17 21:30:03,307 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520602976_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:03,307 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:03,307 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520602976_0001_1
2016-05-17 21:30:03,308 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602976_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520602976_0001_1
2016-05-17 21:30:03,308 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 9E8346D42036427CBB08CC2AD1E3F428 for vertex: vertex_1463520602976_0001_1_00 [CA0AB2B28D7D416CA0638356633496E7]
2016-05-17 21:30:03,313 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520602976_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:03,313 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520602976_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:03,314 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520602976_0001_00_000001, Exiting
2016-05-17 21:30:03,314 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520602976_0001_00_000001
2016-05-17 21:30:03,314 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520602976_0001_00_000001
2016-05-17 21:30:03,316 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520602976_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:03,317 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520602976_0001_00_000001 completed successfully
2016-05-17 21:30:03,322 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520602976_0001_1, queueSize=0
2016-05-17 21:30:03,323 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:03,323 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520602976_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520602976_0001_1, startTime=1463520603176, finishTime=1463520603321, timeTaken=145, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520603285, Process_Duration=6, Process_End_Time=1463520603291, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:03,323 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520602976_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:03,323 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520602976_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:03,324 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520602976_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:03,324 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520602976_0001_00_000001
2016-05-17 21:30:03 Completed Dag: dag_1463520602976_0001_1
2016-05-17 21:30:03,324 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520602976_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:03,325 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520602976_0001_1_post
2016-05-17 21:30:03,325 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:03,325 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:03,332 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, applicationId=application_1463520602976_0001
2016-05-17 21:30:03,334 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:03,334 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:03,334 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:03,335 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:03,335 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:03,335 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:03,336 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:03,336 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:03,336 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:03,336 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:03,336 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:03,337 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:03,337 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:03,342 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FBFBE1E9423F4AC78C47B38BEEA929C2/B272B4897B464A2D84E66B630B5F0035/.tez/application_1463520602976_0001
2016-05-17 21:30:03,343 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1/_temporary
2016-05-17 21:30:03,344 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.390
2016-05-17 21:30:03,421 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_-1
2016-05-17 21:30:03,444 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:03,444 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:03,480 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-05-17 21:30:03,493 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.016
2016-05-17 21:30:03,493 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:03,493 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:03,494 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_'field2'
2016-05-17 21:30:03,532 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:03,532 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:03,609 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-17 21:30:03,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.033
2016-05-17 21:30:03,627 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:03,627 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:03,627 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:03,646 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:03,647 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'
2016-05-17 21:30:03,661 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:03,662 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:03,663 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'"]
2016-05-17 21:30:03,663 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:03,663 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:03,663 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:03,664 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-17 21:30:03,664 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:03,686 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:03,693 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:03,693 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:03,694 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:03,705 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D989859E006540ACB7ABE652C1652FCB/D765B96CCFE44605BCB544247E81FEDE/.tez/application_1463520603693_0001 doesn't exist and is created
2016-05-17 21:30:03,724 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:03,724 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:03,725 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D989859E006540ACB7ABE652C1652FCB/D765B96CCFE44605BCB544247E81FEDE/.tez/application_1463520603693_0001
2016-05-17 21:30:03,729 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520603693_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:03,744 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:03,746 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:03,749 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:03,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:03,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D989859E006540ACB7ABE652C1652FCB/D765B96CCFE44605BCB544247E81FEDE/.tez/application_1463520603693_0001/localmode-log-dir
2016-05-17 21:30:03,751 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D989859E006540ACB7ABE652C1652FCB/D765B96CCFE44605BCB544247E81FEDE/.tez/application_1463520603693_0001/localmode-log-dir/history.txt.appattempt_1463520603693_0001_000000, maxErrors=10
2016-05-17 21:30:03,751 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:03,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520603693_0001_000000, appSubmitTime=1463520603729, launchTime=1463520603729
2016-05-17 21:30:03,759 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:03,761 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:03,762 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:03,763 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:03,770 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:44884
2016-05-17 21:30:03,772 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:03,777 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520603693_0001_000000, startTime=1463520603777
2016-05-17 21:30:03,777 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:03,777 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:03,824 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:03,824 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:03,825 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_'field', applicationId=application_1463520603693_0001, dagName=[D989859E006540ACB7ABE652C1652FCB/D765B96CCFE44605BCB544247E81FEDE] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-17 21:30:03,875 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D989859E006540ACB7ABE652C1652FCB/D765B96CCFE44605BCB544247E81FEDE] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-17 21:30:03,877 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520603693_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D989859E006540ACB7ABE652C1652FCB/D765B96CCFE44605BCB544247E81FEDE/.tez/application_1463520603693_0001/localmode-log-dir/dag_1463520603693_0001_1.dot
2016-05-17 21:30:03,878 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D989859E006540ACB7ABE652C1652FCB/D765B96CCFE44605BCB544247E81FEDE/.tez/application_1463520603693_0001/localmode-log-dir/dag_1463520603693_0001_1-tez-dag.pb.txt
2016-05-17 21:30:03,885 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520603693_0001_1
2016-05-17 21:30:03,886 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:03,886 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D989859E006540ACB7ABE652C1652FCB/D765B96CCFE44605BCB544247E81FEDE] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-17 21:30:03 Running Dag: dag_1463520603693_0001_1
2016-05-17 21:30:03,899 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520603693_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520603693_0001_1, submitTime=1463520603875
2016-05-17 21:30:03,900 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:03,900 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C192BA350EB44F59A124A3C42854629D
2016-05-17 21:30:03,900 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:03,901 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520603693_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520603693_0001_1, initTime=1463520603900
2016-05-17 21:30:03,901 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:03,901 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520603693_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:03,904 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520603693_0001_1][Event:DAG_STARTED]: dagID=dag_1463520603693_0001_1, startTime=1463520603903
2016-05-17 21:30:03,904 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:03,904 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520603693_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:03,904 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C192BA350EB44F59A124A3C42854629D : {223909765C6B4FCB89A50D9BF0D395BF={InputName=223909765C6B4FCB89A50D9BF0D395BF}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:03,904 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 223909765C6B4FCB89A50D9BF0D395BF, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:03,904 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D]
2016-05-17 21:30:03,905 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D]
2016-05-17 21:30:03,905 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D]
2016-05-17 21:30:03,904 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:03,905 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D]. Starting root input initializers: 1
2016-05-17 21:30:03,906 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:03,906 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 223909765C6B4FCB89A50D9BF0D395BF on vertex vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D]
2016-05-17 21:30:03,906 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_'field', applicationId=application_1463520603693_0001, dagName=[D989859E006540ACB7ABE652C1652FCB/D765B96CCFE44605BCB544247E81FEDE] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-17 21:30:03,910 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:03,910 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 223909765C6B4FCB89A50D9BF0D395BF asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:03,911 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:03,912 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:03,915 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:03,918 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 223909765C6B4FCB89A50D9BF0D395BF on vertex vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D]
2016-05-17 21:30:03,919 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:03,919 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {223909765C6B4FCB89A50D9BF0D395BF=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:03,919 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D] parallelism set to 1
2016-05-17 21:30:03,919 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:03,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520603693_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520603693_0001_1_00, eventCount=1
2016-05-17 21:30:03,921 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D]
2016-05-17 21:30:03,921 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=68E48FEF48234A97B0CDABCF8697C68F, vertexId=vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:03,921 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=68E48FEF48234A97B0CDABCF8697C68F, vertexId=vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D]
2016-05-17 21:30:03,923 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=68E48FEF48234A97B0CDABCF8697C68F, vertexName=C192BA350EB44F59A124A3C42854629D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:03,923 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=68E48FEF48234A97B0CDABCF8697C68F, vertexId=vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D]
2016-05-17 21:30:03,925 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520603693_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C192BA350EB44F59A124A3C42854629D, vertexId=vertex_1463520603693_0001_1_00, initRequestedTime=1463520603904, initedTime=1463520603925, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:03,925 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D] with distanceFromRoot: 0
2016-05-17 21:30:03,925 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:03,926 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C192BA350EB44F59A124A3C42854629D
2016-05-17 21:30:03,926 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D]
2016-05-17 21:30:03,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520603693_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520603693_0001_1_00, startRequestedTime=1463520603926, startedTime=1463520603926
2016-05-17 21:30:03,926 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:03,926 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:03,926 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:03,930 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:03,931 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520603693_0001_1][Event:TASK_STARTED]: vertexName=C192BA350EB44F59A124A3C42854629D, taskId=task_1463520603693_0001_1_00_000000, scheduledTime=1463520603930, launchTime=1463520603930
2016-05-17 21:30:03,932 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520603693_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:03,932 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D]
2016-05-17 21:30:03,934 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D989859E006540ACB7ABE652C1652FCB/D765B96CCFE44605BCB544247E81FEDE] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field', VertexName: C192BA350EB44F59A124A3C42854629D, VertexParallelism: 1, TaskAttemptID:attempt_1463520603693_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=223909765C6B4FCB89A50D9BF0D395BF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=68E48FEF48234A97B0CDABCF8697C68F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:03,934 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520603693_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:03,934 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:03,936 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520603693_0001, with dag id: dag_1463520603693_0001_1
2016-05-17 21:30:03,936 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:03,938 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:03,939 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:03,940 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520603693_0001_00_000001
2016-05-17 21:30:03,941 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520603693_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:03,943 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520603693_0001_00_000001
2016-05-17 21:30:03,944 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:03,945 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520603693_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:03,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:03,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520603693_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520603693_0001_00_000001, launchTime=1463520603946
2016-05-17 21:30:03,947 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520603693_0001_00_000001
2016-05-17 21:30:03,947 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520603693_0001_1_00_000000_0] to container: [container_1463520603693_0001_00_000001]
2016-05-17 21:30:03,948 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520603693_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:03,948 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520603693_0001_1_00_000000_0] started. Is using containerId: [container_1463520603693_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:03,949 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:03,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520603693_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C192BA350EB44F59A124A3C42854629D, taskAttemptId=attempt_1463520603693_0001_1_00_000000_0, startTime=1463520603948, containerId=container_1463520603693_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520603693_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:03,949 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520603693_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:03,949 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520603693_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:03,949 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520603693_0001_00_000001 given task: attempt_1463520603693_0001_1_00_000000_0
2016-05-17 21:30:03,950 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520603693_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520603693_0001_1_00_000000_0
2016-05-17 21:30:03 Starting to run new task attempt: attempt_1463520603693_0001_1_00_000000_0
2016-05-17 21:30:03,951 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520603693_0001_1_00_000000_0
2016-05-17 21:30:03,951 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:03,952 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:03,952 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:03,952 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:03,952 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:03,952 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:03,952 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D989859E006540ACB7ABE652C1652FCB/D765B96CCFE44605BCB544247E81FEDE] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field', VertexName: C192BA350EB44F59A124A3C42854629D, VertexParallelism: 1, TaskAttemptID:attempt_1463520603693_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=223909765C6B4FCB89A50D9BF0D395BF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=68E48FEF48234A97B0CDABCF8697C68F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:03,953 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:03,955 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520603693_0001_1_00_000000_0
2016-05-17 21:30:03,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:03,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:03,959 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520603693_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:03,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=223909765C6B4FCB89A50D9BF0D395BF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:03,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:03,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:03,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 223909765C6B4FCB89A50D9BF0D395BF
2016-05-17 21:30:03,963 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 223909765C6B4FCB89A50D9BF0D395BF
2016-05-17 21:30:03,965 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:03,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=68E48FEF48234A97B0CDABCF8697C68F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:03,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:03,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 68E48FEF48234A97B0CDABCF8697C68F
2016-05-17 21:30:03,970 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:03,971 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 68E48FEF48234A97B0CDABCF8697C68F
2016-05-17 21:30:03,968 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:03,972 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 223909765C6B4FCB89A50D9BF0D395BF
2016-05-17 21:30:03,973 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 223909765C6B4FCB89A50D9BF0D395BF
2016-05-17 21:30:03,980 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C192BA350EB44F59A124A3C42854629D, ordinal: 0
2016-05-17 21:30:03,980 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C192BA350EB44F59A124A3C42854629D, mem on start (mb), free: 309, total: 365, max: 672
2016-05-17 21:30:03,981 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:03,981 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:03,981 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:03,988 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:03,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 68E48FEF48234A97B0CDABCF8697C68F
2016-05-17 21:30:03,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:03,988 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:03,989 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:03,989 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:03,989 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:03,989 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:03,989 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:03,989 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:03,989 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 223909765C6B4FCB89A50D9BF0D395BF, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:03,990 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 68E48FEF48234A97B0CDABCF8697C68F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:03,990 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:03,990 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 223909765C6B4FCB89A50D9BF0D395BF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:03,990 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:03,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:03,990 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 223909765C6B4FCB89A50D9BF0D395BF
2016-05-17 21:30:03,992 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:03,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 223909765C6B4FCB89A50D9BF0D395BF
2016-05-17 21:30:03,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:03,992 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520603693_0001_1_00_000000_0
2016-05-17 21:30:03,996 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:03,997 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 223909765C6B4FCB89A50D9BF0D395BF
2016-05-17 21:30:03,998 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'"], id: 68E48FEF48234A97B0CDABCF8697C68F
2016-05-17 21:30:03,998 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'"]
2016-05-17 21:30:03,998 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:03,999 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:03,999 INFO input.MRInput (MRInput.java:checkAndAwaitRecordReaderInitialization(597)) - Awaiting RecordReader initialization
2016-05-17 21:30:04,001 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:04,002 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@236ba71
2016-05-17 21:30:04,004 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@236ba71
2016-05-17 21:30:04,005 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:04,005 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:04,006 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C192BA350EB44F59A124A3C42854629D, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:04,011 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:04,011 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:04,012 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520603693_0001_1_00_000000_0
2016-05-17 21:30:04,012 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520603693_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:04,013 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520603693_0001_1_00_000000_0
2016-05-17 21:30:04,013 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C192BA350EB44F59A124A3C42854629D, mem on close (mb), free: 307, total: 365, max: 672
2016-05-17 21:30:04 Completed running task attempt: attempt_1463520603693_0001_1_00_000000_0
2016-05-17 21:30:04,015 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520603693_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:04,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383254528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520604006, Process_Duration=7, Process_End_Time=1463520604013, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:04,017 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520603693_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C192BA350EB44F59A124A3C42854629D, taskAttemptId=attempt_1463520603693_0001_1_00_000000_0, startTime=1463520603948, finishTime=1463520604016, timeTaken=68, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383254528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520604006, Process_Duration=7, Process_End_Time=1463520604013, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:04,017 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:04,017 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520603693_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:04,017 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:04,018 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520603693_0001_1_00_000000_0
2016-05-17 21:30:04,018 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:04,018 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:04,019 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:04,020 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520603693_0001_1][Event:TASK_FINISHED]: vertexName=C192BA350EB44F59A124A3C42854629D, taskId=task_1463520603693_0001_1_00_000000, startTime=1463520603948, finishTime=1463520604018, timeTaken=70, status=SUCCEEDED, successfulAttemptID=attempt_1463520603693_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383254528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520604006, Process_Duration=7, Process_End_Time=1463520604013, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:04,020 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520603693_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:04,020 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D] : 1
2016-05-17 21:30:04,020 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D]
2016-05-17 21:30:04,018 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520603693_0001_00_000001
2016-05-17 21:30:04,023 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:04,024 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520603693_0001_1][Event:VERTEX_FINISHED]: vertexName=C192BA350EB44F59A124A3C42854629D, vertexId=vertex_1463520603693_0001_1_00, initRequestedTime=1463520603904, initedTime=1463520603925, startRequestedTime=1463520603926, startedTime=1463520603926, finishTime=1463520604020, timeTaken=94, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383254528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520604006, Process_Duration=7, Process_End_Time=1463520604013, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520603948, firstTasksToStart=[ task_1463520603693_0001_1_00_000000 ], lastTaskFinishTime=1463520604016, lastTasksToFinish=[ task_1463520603693_0001_1_00_000000 ], minTaskDuration=68, maxTaskDuration=68, avgTaskDuration=68.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520603693_0001_1_00_000000 ], longestDurationTasks=[ task_1463520603693_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:04,024 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:04,024 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520603693_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:04,025 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:04,025 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520603693_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520603693_0001_00_000001, stoppedTime=1463520604025, exitStatus=0
2016-05-17 21:30:04,025 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520603693_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:04,026 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:04,026 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520603693_0001_1
2016-05-17 21:30:04,027 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520603693_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520603693_0001_1
2016-05-17 21:30:04,027 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 68E48FEF48234A97B0CDABCF8697C68F for vertex: vertex_1463520603693_0001_1_00 [C192BA350EB44F59A124A3C42854629D]
2016-05-17 21:30:04,031 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520603693_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:04,031 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520603693_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:04,031 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520603693_0001_00_000001, Exiting
2016-05-17 21:30:04,031 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520603693_0001_00_000001
2016-05-17 21:30:04,031 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520603693_0001_00_000001
2016-05-17 21:30:04,033 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520603693_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:04,034 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520603693_0001_00_000001 completed successfully
2016-05-17 21:30:04,037 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520603693_0001_1, queueSize=0
2016-05-17 21:30:04,037 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:04,038 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520603693_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520603693_0001_1, startTime=1463520603903, finishTime=1463520604035, timeTaken=132, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383254528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520604006, Process_Duration=7, Process_End_Time=1463520604013, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:04,038 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520603693_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:04,038 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520603693_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:04,039 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520603693_0001_00_000001
2016-05-17 21:30:04,039 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520603693_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:04 Completed Dag: dag_1463520603693_0001_1
2016-05-17 21:30:04,040 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520603693_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:04,040 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520603693_0001_1_post
2016-05-17 21:30:04,041 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:04,041 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:04,045 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_'field', applicationId=application_1463520603693_0001
2016-05-17 21:30:04,049 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:04,049 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:04,049 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:04,049 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:04,050 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:04,050 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:04,050 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:04,052 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:04,052 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:04,052 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:04,052 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:04,052 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:04,053 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:04,058 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D989859E006540ACB7ABE652C1652FCB/D765B96CCFE44605BCB544247E81FEDE/.tez/application_1463520603693_0001
2016-05-17 21:30:04,058 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'/_temporary
2016-05-17 21:30:04,059 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.397
2016-05-17 21:30:04,133 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_'field'
2016-05-17 21:30:04,157 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:04,158 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:04,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-17 21:30:04,207 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-17 21:30:04,208 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:04,208 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:04,208 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:04,223 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:04,224 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS
2016-05-17 21:30:04,235 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:04,236 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:04,236 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS"]
2016-05-17 21:30:04,236 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:04,236 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:04,236 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:04,238 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-17 21:30:04,238 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:04,250 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:04,255 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:04,255 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:04,255 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:04,265 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FB51728DBD31499AB2E335D0E5FCD0D0/F5243B4E49EA4ECF96066121D32EC6E3/.tez/application_1463520604255_0001 doesn't exist and is created
2016-05-17 21:30:04,283 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:04,283 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:04,284 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FB51728DBD31499AB2E335D0E5FCD0D0/F5243B4E49EA4ECF96066121D32EC6E3/.tez/application_1463520604255_0001
2016-05-17 21:30:04,288 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520604255_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:04,300 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:04,301 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:04,303 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:04,303 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:04,304 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FB51728DBD31499AB2E335D0E5FCD0D0/F5243B4E49EA4ECF96066121D32EC6E3/.tez/application_1463520604255_0001/localmode-log-dir
2016-05-17 21:30:04,304 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FB51728DBD31499AB2E335D0E5FCD0D0/F5243B4E49EA4ECF96066121D32EC6E3/.tez/application_1463520604255_0001/localmode-log-dir/history.txt.appattempt_1463520604255_0001_000000, maxErrors=10
2016-05-17 21:30:04,304 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:04,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520604255_0001_000000, appSubmitTime=1463520604287, launchTime=1463520604287
2016-05-17 21:30:04,308 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:04,310 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:04,312 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:04,313 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:04,315 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56041
2016-05-17 21:30:04,317 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:04,320 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520604255_0001_000000, startTime=1463520604320
2016-05-17 21:30:04,320 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:04,320 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:04,383 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:04,383 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:04,384 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, applicationId=application_1463520604255_0001, dagName=[FB51728DBD31499AB2E335D0E5FCD0D0/F5243B4E49EA4ECF96066121D32EC6E3] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-17 21:30:04,400 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [FB51728DBD31499AB2E335D0E5FCD0D0/F5243B4E49EA4ECF96066121D32EC6E3] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-17 21:30:04,401 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520604255_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FB51728DBD31499AB2E335D0E5FCD0D0/F5243B4E49EA4ECF96066121D32EC6E3/.tez/application_1463520604255_0001/localmode-log-dir/dag_1463520604255_0001_1.dot
2016-05-17 21:30:04,402 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FB51728DBD31499AB2E335D0E5FCD0D0/F5243B4E49EA4ECF96066121D32EC6E3/.tez/application_1463520604255_0001/localmode-log-dir/dag_1463520604255_0001_1-tez-dag.pb.txt
2016-05-17 21:30:04,413 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520604255_0001_1
2016-05-17 21:30:04,413 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:04,414 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [FB51728DBD31499AB2E335D0E5FCD0D0/F5243B4E49EA4ECF96066121D32EC6E3] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-17 21:30:04 Running Dag: dag_1463520604255_0001_1
2016-05-17 21:30:04,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604255_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520604255_0001_1, submitTime=1463520604400
2016-05-17 21:30:04,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:04,426 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BAC808A45C88463781D2A84803E91EF2
2016-05-17 21:30:04,427 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:04,427 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:04,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604255_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520604255_0001_1, initTime=1463520604426
2016-05-17 21:30:04,428 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520604255_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:04,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604255_0001_1][Event:DAG_STARTED]: dagID=dag_1463520604255_0001_1, startTime=1463520604429
2016-05-17 21:30:04,429 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:04,429 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520604255_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:04,429 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BAC808A45C88463781D2A84803E91EF2 : {E8A7E74396BA4CE6BE3ED730B4A64044={InputName=E8A7E74396BA4CE6BE3ED730B4A64044}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:04,429 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E8A7E74396BA4CE6BE3ED730B4A64044, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:04,429 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2]
2016-05-17 21:30:04,430 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2]
2016-05-17 21:30:04,430 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2]
2016-05-17 21:30:04,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:04,430 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2]. Starting root input initializers: 1
2016-05-17 21:30:04,430 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:04,431 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E8A7E74396BA4CE6BE3ED730B4A64044 on vertex vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2]
2016-05-17 21:30:04,431 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, applicationId=application_1463520604255_0001, dagName=[FB51728DBD31499AB2E335D0E5FCD0D0/F5243B4E49EA4ECF96066121D32EC6E3] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-17 21:30:04,432 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:04,432 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E8A7E74396BA4CE6BE3ED730B4A64044 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:04,433 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:04,433 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:04,435 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:04,436 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E8A7E74396BA4CE6BE3ED730B4A64044 on vertex vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2]
2016-05-17 21:30:04,436 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:04,436 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E8A7E74396BA4CE6BE3ED730B4A64044=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:04,437 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2] parallelism set to 1
2016-05-17 21:30:04,437 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:04,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604255_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520604255_0001_1_00, eventCount=1
2016-05-17 21:30:04,437 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2]
2016-05-17 21:30:04,437 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=72D840E62ECC4E5DB23D73FF1AD61CCF, vertexId=vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:04,438 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=72D840E62ECC4E5DB23D73FF1AD61CCF, vertexId=vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2]
2016-05-17 21:30:04,440 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=72D840E62ECC4E5DB23D73FF1AD61CCF, vertexName=BAC808A45C88463781D2A84803E91EF2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:04,440 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=72D840E62ECC4E5DB23D73FF1AD61CCF, vertexId=vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2]
2016-05-17 21:30:04,441 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604255_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BAC808A45C88463781D2A84803E91EF2, vertexId=vertex_1463520604255_0001_1_00, initRequestedTime=1463520604429, initedTime=1463520604441, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:04,441 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:04,442 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2] with distanceFromRoot: 0
2016-05-17 21:30:04,442 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:04,445 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BAC808A45C88463781D2A84803E91EF2
2016-05-17 21:30:04,445 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2]
2016-05-17 21:30:04,445 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604255_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520604255_0001_1_00, startRequestedTime=1463520604442, startedTime=1463520604442
2016-05-17 21:30:04,445 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:04,446 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:04,447 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:04,447 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604255_0001_1][Event:TASK_STARTED]: vertexName=BAC808A45C88463781D2A84803E91EF2, taskId=task_1463520604255_0001_1_00_000000, scheduledTime=1463520604446, launchTime=1463520604446
2016-05-17 21:30:04,447 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520604255_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:04,448 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2]
2016-05-17 21:30:04,448 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FB51728DBD31499AB2E335D0E5FCD0D0/F5243B4E49EA4ECF96066121D32EC6E3] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, VertexName: BAC808A45C88463781D2A84803E91EF2, VertexParallelism: 1, TaskAttemptID:attempt_1463520604255_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E8A7E74396BA4CE6BE3ED730B4A64044, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=72D840E62ECC4E5DB23D73FF1AD61CCF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:04,449 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520604255_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:04,449 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:04,450 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:04,451 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:04,451 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:04,452 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520604255_0001, with dag id: dag_1463520604255_0001_1
2016-05-17 21:30:04,452 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520604255_0001_00_000001
2016-05-17 21:30:04,453 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520604255_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:04,456 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520604255_0001_00_000001
2016-05-17 21:30:04,456 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:04,457 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520604255_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:04,457 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:04,457 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604255_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520604255_0001_00_000001, launchTime=1463520604457
2016-05-17 21:30:04,458 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520604255_0001_00_000001
2016-05-17 21:30:04,460 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520604255_0001_1_00_000000_0] to container: [container_1463520604255_0001_00_000001]
2016-05-17 21:30:04,460 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520604255_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:04,461 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520604255_0001_00_000001 given task: attempt_1463520604255_0001_1_00_000000_0
2016-05-17 21:30:04,461 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520604255_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520604255_0001_1_00_000000_0
2016-05-17 21:30:04,461 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520604255_0001_1_00_000000_0] started. Is using containerId: [container_1463520604255_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:04,462 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:04,462 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604255_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BAC808A45C88463781D2A84803E91EF2, taskAttemptId=attempt_1463520604255_0001_1_00_000000_0, startTime=1463520604461, containerId=container_1463520604255_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520604255_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:04,462 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520604255_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:04,462 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520604255_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:04 Starting to run new task attempt: attempt_1463520604255_0001_1_00_000000_0
2016-05-17 21:30:04,463 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520604255_0001_1_00_000000_0
2016-05-17 21:30:04,464 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:04,464 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:04,464 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:04,465 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:04,465 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:04,465 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:04,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FB51728DBD31499AB2E335D0E5FCD0D0/F5243B4E49EA4ECF96066121D32EC6E3] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, VertexName: BAC808A45C88463781D2A84803E91EF2, VertexParallelism: 1, TaskAttemptID:attempt_1463520604255_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E8A7E74396BA4CE6BE3ED730B4A64044, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=72D840E62ECC4E5DB23D73FF1AD61CCF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:04,466 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:04,466 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520604255_0001_1_00_000000_0
2016-05-17 21:30:04,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:04,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:04,467 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520604255_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:04,468 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:04,472 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:04,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=72D840E62ECC4E5DB23D73FF1AD61CCF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:04,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E8A7E74396BA4CE6BE3ED730B4A64044, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:04,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:04,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E8A7E74396BA4CE6BE3ED730B4A64044
2016-05-17 21:30:04,475 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E8A7E74396BA4CE6BE3ED730B4A64044
2016-05-17 21:30:04,475 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:04,475 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 72D840E62ECC4E5DB23D73FF1AD61CCF
2016-05-17 21:30:04,476 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:04,476 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 72D840E62ECC4E5DB23D73FF1AD61CCF
2016-05-17 21:30:04,478 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:04,478 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E8A7E74396BA4CE6BE3ED730B4A64044
2016-05-17 21:30:04,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E8A7E74396BA4CE6BE3ED730B4A64044
2016-05-17 21:30:04,482 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BAC808A45C88463781D2A84803E91EF2, ordinal: 0
2016-05-17 21:30:04,482 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BAC808A45C88463781D2A84803E91EF2, mem on start (mb), free: 231, total: 365, max: 672
2016-05-17 21:30:04,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:04,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:04,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:04,489 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:04,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 72D840E62ECC4E5DB23D73FF1AD61CCF
2016-05-17 21:30:04,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:04,489 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:04,489 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:04,489 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:04,489 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:04,490 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:04,490 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:04,490 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:04,490 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E8A7E74396BA4CE6BE3ED730B4A64044, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:04,490 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 72D840E62ECC4E5DB23D73FF1AD61CCF, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:04,490 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:04,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E8A7E74396BA4CE6BE3ED730B4A64044 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:04,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:04,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:04,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E8A7E74396BA4CE6BE3ED730B4A64044
2016-05-17 21:30:04,492 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:04,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E8A7E74396BA4CE6BE3ED730B4A64044
2016-05-17 21:30:04,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:04,492 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520604255_0001_1_00_000000_0
2016-05-17 21:30:04,493 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:04,494 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3a260214
2016-05-17 21:30:04,495 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:04,495 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3a260214
2016-05-17 21:30:04,495 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:04,496 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:04,495 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E8A7E74396BA4CE6BE3ED730B4A64044
2016-05-17 21:30:04,496 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS"], id: 72D840E62ECC4E5DB23D73FF1AD61CCF
2016-05-17 21:30:04,496 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS"]
2016-05-17 21:30:04,497 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:04,497 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:04,497 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BAC808A45C88463781D2A84803E91EF2, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:04,499 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:04,500 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:04,500 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520604255_0001_1_00_000000_0
2016-05-17 21:30:04,500 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520604255_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:04,501 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520604255_0001_1_00_000000_0
2016-05-17 21:30:04,501 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BAC808A45C88463781D2A84803E91EF2, mem on close (mb), free: 230, total: 365, max: 672
2016-05-17 21:30:04 Completed running task attempt: attempt_1463520604255_0001_1_00_000000_0
2016-05-17 21:30:04,502 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520604255_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:04,503 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383254528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520604497, Process_Duration=4, Process_End_Time=1463520604501, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:04,503 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:04,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604255_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BAC808A45C88463781D2A84803E91EF2, taskAttemptId=attempt_1463520604255_0001_1_00_000000_0, startTime=1463520604461, finishTime=1463520604503, timeTaken=42, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383254528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520604497, Process_Duration=4, Process_End_Time=1463520604501, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:04,503 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520604255_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:04,503 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520604255_0001_1_00_000000_0
2016-05-17 21:30:04,504 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:04,504 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:04,504 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:04,505 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:04,504 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604255_0001_1][Event:TASK_FINISHED]: vertexName=BAC808A45C88463781D2A84803E91EF2, taskId=task_1463520604255_0001_1_00_000000, startTime=1463520604461, finishTime=1463520604503, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463520604255_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383254528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520604497, Process_Duration=4, Process_End_Time=1463520604501, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:04,506 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520604255_0001_00_000001
2016-05-17 21:30:04,506 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520604255_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:04,507 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2] : 1
2016-05-17 21:30:04,507 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2]
2016-05-17 21:30:04,508 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:04,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604255_0001_1][Event:VERTEX_FINISHED]: vertexName=BAC808A45C88463781D2A84803E91EF2, vertexId=vertex_1463520604255_0001_1_00, initRequestedTime=1463520604429, initedTime=1463520604441, startRequestedTime=1463520604442, startedTime=1463520604442, finishTime=1463520604507, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383254528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520604497, Process_Duration=4, Process_End_Time=1463520604501, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520604461, firstTasksToStart=[ task_1463520604255_0001_1_00_000000 ], lastTaskFinishTime=1463520604503, lastTasksToFinish=[ task_1463520604255_0001_1_00_000000 ], minTaskDuration=42, maxTaskDuration=42, avgTaskDuration=42.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520604255_0001_1_00_000000 ], longestDurationTasks=[ task_1463520604255_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:04,509 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:04,509 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520604255_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:04,509 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604255_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520604255_0001_00_000001, stoppedTime=1463520604509, exitStatus=0
2016-05-17 21:30:04,509 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:04,509 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520604255_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:04,510 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:04,510 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520604255_0001_1
2016-05-17 21:30:04,510 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604255_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520604255_0001_1
2016-05-17 21:30:04,511 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 72D840E62ECC4E5DB23D73FF1AD61CCF for vertex: vertex_1463520604255_0001_1_00 [BAC808A45C88463781D2A84803E91EF2]
2016-05-17 21:30:04,517 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520604255_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:04,517 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520604255_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:04,517 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520604255_0001_00_000001, Exiting
2016-05-17 21:30:04,517 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520604255_0001_00_000001
2016-05-17 21:30:04,517 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520604255_0001_00_000001
2016-05-17 21:30:04,520 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520604255_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:04,520 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520604255_0001_00_000001 completed successfully
2016-05-17 21:30:04,521 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520604255_0001_1, queueSize=0
2016-05-17 21:30:04,522 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:04,522 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604255_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520604255_0001_1, startTime=1463520604429, finishTime=1463520604520, timeTaken=91, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383254528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520604497, Process_Duration=4, Process_End_Time=1463520604501, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:04,523 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520604255_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:04,523 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520604255_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:04,524 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520604255_0001_00_000001
2016-05-17 21:30:04,524 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520604255_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:04 Completed Dag: dag_1463520604255_0001_1
2016-05-17 21:30:04,525 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520604255_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:04,525 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520604255_0001_1_post
2016-05-17 21:30:04,526 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:04,526 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:04,531 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, applicationId=application_1463520604255_0001
2016-05-17 21:30:04,532 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:04,533 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:04,533 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:04,533 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:04,533 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:04,533 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:04,534 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:04,534 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:04,534 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:04,535 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:04,536 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:04,536 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:04,536 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:04,540 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FB51728DBD31499AB2E335D0E5FCD0D0/F5243B4E49EA4ECF96066121D32EC6E3/.tez/application_1463520604255_0001
2016-05-17 21:30:04,540 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS/_temporary
2016-05-17 21:30:04,541 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.305
2016-05-17 21:30:04,611 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_RESULTS
2016-05-17 21:30:04,635 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:04,636 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:04,681 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-17 21:30:04,687 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-05-17 21:30:04,687 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:04,687 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:04,687 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:04,702 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:04,703 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL
2016-05-17 21:30:04,715 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:04,715 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:04,716 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL"]
2016-05-17 21:30:04,716 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:04,716 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:04,716 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:04,717 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-17 21:30:04,717 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:04,732 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:04,737 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:04,737 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:04,737 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:04,747 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2A613942B493496B88754EB4488BCBB3/CEABF70591AA46AC80CDACCC39806C07/.tez/application_1463520604737_0001 doesn't exist and is created
2016-05-17 21:30:04,764 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:04,764 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:04,765 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2A613942B493496B88754EB4488BCBB3/CEABF70591AA46AC80CDACCC39806C07/.tez/application_1463520604737_0001
2016-05-17 21:30:04,769 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520604737_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:04,782 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:04,783 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:04,785 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:04,785 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:04,785 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2A613942B493496B88754EB4488BCBB3/CEABF70591AA46AC80CDACCC39806C07/.tez/application_1463520604737_0001/localmode-log-dir
2016-05-17 21:30:04,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2A613942B493496B88754EB4488BCBB3/CEABF70591AA46AC80CDACCC39806C07/.tez/application_1463520604737_0001/localmode-log-dir/history.txt.appattempt_1463520604737_0001_000000, maxErrors=10
2016-05-17 21:30:04,786 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:04,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520604737_0001_000000, appSubmitTime=1463520604768, launchTime=1463520604768
2016-05-17 21:30:04,790 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:04,791 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:04,795 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41082
2016-05-17 21:30:04,797 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:04,797 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:04,798 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:04,800 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520604737_0001_000000, startTime=1463520604800
2016-05-17 21:30:04,800 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:04,800 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:04,865 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:04,865 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:04,865 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, applicationId=application_1463520604737_0001, dagName=[2A613942B493496B88754EB4488BCBB3/CEABF70591AA46AC80CDACCC39806C07] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-17 21:30:04,882 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2A613942B493496B88754EB4488BCBB3/CEABF70591AA46AC80CDACCC39806C07] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-17 21:30:04,883 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520604737_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2A613942B493496B88754EB4488BCBB3/CEABF70591AA46AC80CDACCC39806C07/.tez/application_1463520604737_0001/localmode-log-dir/dag_1463520604737_0001_1.dot
2016-05-17 21:30:04,884 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2A613942B493496B88754EB4488BCBB3/CEABF70591AA46AC80CDACCC39806C07/.tez/application_1463520604737_0001/localmode-log-dir/dag_1463520604737_0001_1-tez-dag.pb.txt
2016-05-17 21:30:04,890 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520604737_0001_1
2016-05-17 21:30:04,890 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:04,890 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2A613942B493496B88754EB4488BCBB3/CEABF70591AA46AC80CDACCC39806C07] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-17 21:30:04 Running Dag: dag_1463520604737_0001_1
2016-05-17 21:30:04,904 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604737_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520604737_0001_1, submitTime=1463520604882
2016-05-17 21:30:04,904 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:04,904 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BC5C6906B91D42CAA38F2595B255E67E
2016-05-17 21:30:04,905 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:04,905 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604737_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520604737_0001_1, initTime=1463520604904
2016-05-17 21:30:04,905 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:04,906 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520604737_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:04,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604737_0001_1][Event:DAG_STARTED]: dagID=dag_1463520604737_0001_1, startTime=1463520604906
2016-05-17 21:30:04,907 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:04,907 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520604737_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:04,907 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BC5C6906B91D42CAA38F2595B255E67E : {B1DFE03633B643FD825550D65616E3D4={InputName=B1DFE03633B643FD825550D65616E3D4}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:04,907 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B1DFE03633B643FD825550D65616E3D4, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:04,908 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E]
2016-05-17 21:30:04,908 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E]
2016-05-17 21:30:04,908 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E]
2016-05-17 21:30:04,908 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E]. Starting root input initializers: 1
2016-05-17 21:30:04,907 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:04,909 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:04,909 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B1DFE03633B643FD825550D65616E3D4 on vertex vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E]
2016-05-17 21:30:04,907 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, applicationId=application_1463520604737_0001, dagName=[2A613942B493496B88754EB4488BCBB3/CEABF70591AA46AC80CDACCC39806C07] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-17 21:30:04,910 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:04,911 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B1DFE03633B643FD825550D65616E3D4 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:04,911 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:04,911 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:04,913 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:04,913 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B1DFE03633B643FD825550D65616E3D4 on vertex vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E]
2016-05-17 21:30:04,913 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:04,913 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B1DFE03633B643FD825550D65616E3D4=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:04,914 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E] parallelism set to 1
2016-05-17 21:30:04,914 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:04,914 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604737_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520604737_0001_1_00, eventCount=1
2016-05-17 21:30:04,914 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E]
2016-05-17 21:30:04,914 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=5E85974BFA884D6E8883A4A441D4DB82, vertexId=vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:04,915 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=5E85974BFA884D6E8883A4A441D4DB82, vertexId=vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E]
2016-05-17 21:30:04,916 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=5E85974BFA884D6E8883A4A441D4DB82, vertexName=BC5C6906B91D42CAA38F2595B255E67E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:04,916 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=5E85974BFA884D6E8883A4A441D4DB82, vertexId=vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E]
2016-05-17 21:30:04,918 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:04,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604737_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BC5C6906B91D42CAA38F2595B255E67E, vertexId=vertex_1463520604737_0001_1_00, initRequestedTime=1463520604907, initedTime=1463520604917, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:04,918 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E] with distanceFromRoot: 0
2016-05-17 21:30:04,918 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:04,919 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BC5C6906B91D42CAA38F2595B255E67E
2016-05-17 21:30:04,919 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E]
2016-05-17 21:30:04,919 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:04,919 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604737_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520604737_0001_1_00, startRequestedTime=1463520604918, startedTime=1463520604919
2016-05-17 21:30:04,919 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:04,920 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:04,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604737_0001_1][Event:TASK_STARTED]: vertexName=BC5C6906B91D42CAA38F2595B255E67E, taskId=task_1463520604737_0001_1_00_000000, scheduledTime=1463520604920, launchTime=1463520604920
2016-05-17 21:30:04,920 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520604737_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:04,920 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E]
2016-05-17 21:30:04,921 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2A613942B493496B88754EB4488BCBB3/CEABF70591AA46AC80CDACCC39806C07] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, VertexName: BC5C6906B91D42CAA38F2595B255E67E, VertexParallelism: 1, TaskAttemptID:attempt_1463520604737_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B1DFE03633B643FD825550D65616E3D4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5E85974BFA884D6E8883A4A441D4DB82, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:04,921 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520604737_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:04,921 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:04,922 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:04,922 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:04,922 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:04,923 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520604737_0001_00_000001
2016-05-17 21:30:04,923 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520604737_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:04,925 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520604737_0001_00_000001
2016-05-17 21:30:04,925 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:04,926 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:04,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604737_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520604737_0001_00_000001, launchTime=1463520604926
2016-05-17 21:30:04,926 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520604737_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:04,927 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520604737_0001_00_000001
2016-05-17 21:30:04,929 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520604737_0001_1_00_000000_0] to container: [container_1463520604737_0001_00_000001]
2016-05-17 21:30:04,930 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520604737_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:04,930 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520604737_0001_00_000001 given task: attempt_1463520604737_0001_1_00_000000_0
2016-05-17 21:30:04,930 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520604737_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520604737_0001_1_00_000000_0
2016-05-17 21:30:04,930 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520604737_0001_1_00_000000_0] started. Is using containerId: [container_1463520604737_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:04,930 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:04,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604737_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BC5C6906B91D42CAA38F2595B255E67E, taskAttemptId=attempt_1463520604737_0001_1_00_000000_0, startTime=1463520604930, containerId=container_1463520604737_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520604737_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:04 Starting to run new task attempt: attempt_1463520604737_0001_1_00_000000_0
2016-05-17 21:30:04,931 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520604737_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:04,931 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520604737_0001_1_00_000000_0
2016-05-17 21:30:04,932 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:04,932 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:04,932 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:04,932 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:04,932 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:04,933 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:04,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2A613942B493496B88754EB4488BCBB3/CEABF70591AA46AC80CDACCC39806C07] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, VertexName: BC5C6906B91D42CAA38F2595B255E67E, VertexParallelism: 1, TaskAttemptID:attempt_1463520604737_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B1DFE03633B643FD825550D65616E3D4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5E85974BFA884D6E8883A4A441D4DB82, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:04,933 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:04,933 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520604737_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:04,934 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520604737_0001, with dag id: dag_1463520604737_0001_1
2016-05-17 21:30:04,937 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520604737_0001_1_00_000000_0
2016-05-17 21:30:04,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:04,937 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520604737_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:04,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:04,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:04,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B1DFE03633B643FD825550D65616E3D4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:04,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:04,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B1DFE03633B643FD825550D65616E3D4
2016-05-17 21:30:04,939 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B1DFE03633B643FD825550D65616E3D4
2016-05-17 21:30:04,941 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:04,941 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:04,942 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B1DFE03633B643FD825550D65616E3D4
2016-05-17 21:30:04,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B1DFE03633B643FD825550D65616E3D4
2016-05-17 21:30:04,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=5E85974BFA884D6E8883A4A441D4DB82, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:04,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:04,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 5E85974BFA884D6E8883A4A441D4DB82
2016-05-17 21:30:04,946 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:04,946 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 5E85974BFA884D6E8883A4A441D4DB82
2016-05-17 21:30:04,956 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BC5C6906B91D42CAA38F2595B255E67E, ordinal: 0
2016-05-17 21:30:04,956 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BC5C6906B91D42CAA38F2595B255E67E, mem on start (mb), free: 157, total: 365, max: 672
2016-05-17 21:30:04,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:04,957 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:04,957 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:04,957 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:04,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 5E85974BFA884D6E8883A4A441D4DB82
2016-05-17 21:30:04,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:04,958 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:04,959 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:04,959 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:04,959 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:04,960 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:04,960 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:04,961 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:04,961 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B1DFE03633B643FD825550D65616E3D4, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:04,961 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 5E85974BFA884D6E8883A4A441D4DB82, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:04,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:04,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B1DFE03633B643FD825550D65616E3D4
2016-05-17 21:30:04,963 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:04,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B1DFE03633B643FD825550D65616E3D4
2016-05-17 21:30:04,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B1DFE03633B643FD825550D65616E3D4 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:04,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:04,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:04,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:04,965 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520604737_0001_1_00_000000_0
2016-05-17 21:30:04,965 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:04,966 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@789f1442
2016-05-17 21:30:04,968 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@789f1442
2016-05-17 21:30:04,968 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:04,968 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:04,971 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:04,971 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B1DFE03633B643FD825550D65616E3D4
2016-05-17 21:30:04,972 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL"], id: 5E85974BFA884D6E8883A4A441D4DB82
2016-05-17 21:30:04,972 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL"]
2016-05-17 21:30:04,972 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:04,972 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:04,973 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BC5C6906B91D42CAA38F2595B255E67E, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:30:04,976 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:04,977 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:04,977 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520604737_0001_1_00_000000_0
2016-05-17 21:30:04,977 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520604737_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:04,978 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520604737_0001_1_00_000000_0
2016-05-17 21:30:04,978 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BC5C6906B91D42CAA38F2595B255E67E, mem on close (mb), free: 156, total: 365, max: 672
2016-05-17 21:30:04 Completed running task attempt: attempt_1463520604737_0001_1_00_000000_0
2016-05-17 21:30:04,979 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520604737_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:04,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383254528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520604973, Process_Duration=5, Process_End_Time=1463520604978, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:04,980 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604737_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BC5C6906B91D42CAA38F2595B255E67E, taskAttemptId=attempt_1463520604737_0001_1_00_000000_0, startTime=1463520604930, finishTime=1463520604980, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383254528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520604973, Process_Duration=5, Process_End_Time=1463520604978, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:04,980 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:04,981 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520604737_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:04,981 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:04,982 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:04,982 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520604737_0001_1_00_000000_0
2016-05-17 21:30:04,983 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:04,983 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520604737_0001_00_000001
2016-05-17 21:30:04,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604737_0001_1][Event:TASK_FINISHED]: vertexName=BC5C6906B91D42CAA38F2595B255E67E, taskId=task_1463520604737_0001_1_00_000000, startTime=1463520604930, finishTime=1463520604983, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1463520604737_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383254528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520604973, Process_Duration=5, Process_End_Time=1463520604978, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:04,983 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:04,984 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520604737_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:04,984 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E] : 1
2016-05-17 21:30:04,984 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E]
2016-05-17 21:30:04,985 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:04,985 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604737_0001_1][Event:VERTEX_FINISHED]: vertexName=BC5C6906B91D42CAA38F2595B255E67E, vertexId=vertex_1463520604737_0001_1_00, initRequestedTime=1463520604907, initedTime=1463520604917, startRequestedTime=1463520604918, startedTime=1463520604919, finishTime=1463520604984, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383254528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520604973, Process_Duration=5, Process_End_Time=1463520604978, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520604930, firstTasksToStart=[ task_1463520604737_0001_1_00_000000 ], lastTaskFinishTime=1463520604980, lastTasksToFinish=[ task_1463520604737_0001_1_00_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520604737_0001_1_00_000000 ], longestDurationTasks=[ task_1463520604737_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:04,986 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:04,987 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520604737_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:04,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604737_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520604737_0001_00_000001, stoppedTime=1463520604988, exitStatus=0
2016-05-17 21:30:04,988 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:04,988 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520604737_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:04,988 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:04,988 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520604737_0001_1
2016-05-17 21:30:04,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604737_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520604737_0001_1
2016-05-17 21:30:04,989 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 5E85974BFA884D6E8883A4A441D4DB82 for vertex: vertex_1463520604737_0001_1_00 [BC5C6906B91D42CAA38F2595B255E67E]
2016-05-17 21:30:04,994 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520604737_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:04,994 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520604737_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:04,994 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520604737_0001_00_000001, Exiting
2016-05-17 21:30:04,994 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520604737_0001_00_000001
2016-05-17 21:30:04,994 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520604737_0001_00_000001
2016-05-17 21:30:04,998 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520604737_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:04,998 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520604737_0001_00_000001 completed successfully
2016-05-17 21:30:04,999 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520604737_0001_1, queueSize=0
2016-05-17 21:30:04,999 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:05,000 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520604737_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520604737_0001_1, startTime=1463520604906, finishTime=1463520604997, timeTaken=91, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383254528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520604973, Process_Duration=5, Process_End_Time=1463520604978, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:05,000 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520604737_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:05,000 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520604737_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:05,000 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520604737_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:05,000 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520604737_0001_00_000001
2016-05-17 21:30:05 Completed Dag: dag_1463520604737_0001_1
2016-05-17 21:30:05,003 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520604737_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:05,003 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520604737_0001_1_post
2016-05-17 21:30:05,004 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:05,004 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:05,007 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, applicationId=application_1463520604737_0001
2016-05-17 21:30:05,009 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:05,009 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:05,009 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:05,009 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:05,010 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:05,010 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:05,010 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:05,012 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:05,012 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:05,012 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:05,012 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:05,012 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:05,013 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:05,018 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2A613942B493496B88754EB4488BCBB3/CEABF70591AA46AC80CDACCC39806C07/.tez/application_1463520604737_0001
2016-05-17 21:30:05,019 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL/_temporary
2016-05-17 21:30:05,019 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.303
2016-05-17 21:30:05,091 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_ALL
2016-05-17 21:30:05,106 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:05,106 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:05,124 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:05,124 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:05,179 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:30:05,193 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-17 21:30:05,194 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:05,194 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:05,194 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:05,209 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:05,210 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1
2016-05-17 21:30:05,220 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:05,220 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:05,220 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1"]
2016-05-17 21:30:05,220 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:05,221 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:05,221 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:05,222 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-17 21:30:05,222 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:05,235 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:05,239 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:05,239 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:05,239 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:05,249 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/550FF659B5B9423299D956DAA0716B68/ED65CA08069B4853A183B14A35F22D41/.tez/application_1463520605239_0001 doesn't exist and is created
2016-05-17 21:30:05,265 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:05,265 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:05,265 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/550FF659B5B9423299D956DAA0716B68/ED65CA08069B4853A183B14A35F22D41/.tez/application_1463520605239_0001
2016-05-17 21:30:05,269 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520605239_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:05,281 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:05,281 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:05,283 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:05,283 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:05,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/550FF659B5B9423299D956DAA0716B68/ED65CA08069B4853A183B14A35F22D41/.tez/application_1463520605239_0001/localmode-log-dir
2016-05-17 21:30:05,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/550FF659B5B9423299D956DAA0716B68/ED65CA08069B4853A183B14A35F22D41/.tez/application_1463520605239_0001/localmode-log-dir/history.txt.appattempt_1463520605239_0001_000000, maxErrors=10
2016-05-17 21:30:05,284 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:05,284 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520605239_0001_000000, appSubmitTime=1463520605269, launchTime=1463520605269
2016-05-17 21:30:05,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:05,291 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:05,293 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:51292
2016-05-17 21:30:05,295 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:05,295 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:05,295 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:05,297 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520605239_0001_000000, startTime=1463520605297
2016-05-17 21:30:05,298 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:05,297 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:05,365 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:05,366 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:05,366 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.-1_VALUES_-1, applicationId=application_1463520605239_0001, dagName=[550FF659B5B9423299D956DAA0716B68/ED65CA08069B4853A183B14A35F22D41] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-17 21:30:05,381 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [550FF659B5B9423299D956DAA0716B68/ED65CA08069B4853A183B14A35F22D41] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-17 21:30:05,383 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520605239_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/550FF659B5B9423299D956DAA0716B68/ED65CA08069B4853A183B14A35F22D41/.tez/application_1463520605239_0001/localmode-log-dir/dag_1463520605239_0001_1.dot
2016-05-17 21:30:05,383 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/550FF659B5B9423299D956DAA0716B68/ED65CA08069B4853A183B14A35F22D41/.tez/application_1463520605239_0001/localmode-log-dir/dag_1463520605239_0001_1-tez-dag.pb.txt
2016-05-17 21:30:05,389 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520605239_0001_1
2016-05-17 21:30:05,389 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:05,390 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [550FF659B5B9423299D956DAA0716B68/ED65CA08069B4853A183B14A35F22D41] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-17 21:30:05 Running Dag: dag_1463520605239_0001_1
2016-05-17 21:30:05,402 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605239_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520605239_0001_1, submitTime=1463520605382
2016-05-17 21:30:05,402 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:05,403 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C31D863F05CD49889131C51F482CDA1F
2016-05-17 21:30:05,403 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:05,404 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605239_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520605239_0001_1, initTime=1463520605402
2016-05-17 21:30:05,404 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:05,404 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520605239_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:05,404 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605239_0001_1][Event:DAG_STARTED]: dagID=dag_1463520605239_0001_1, startTime=1463520605404
2016-05-17 21:30:05,405 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:05,405 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520605239_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:05,405 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:05,405 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C31D863F05CD49889131C51F482CDA1F : {45465605354D45649984A749A5CA4DDE={InputName=45465605354D45649984A749A5CA4DDE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:05,407 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 45465605354D45649984A749A5CA4DDE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:05,407 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F]
2016-05-17 21:30:05,406 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.-1_VALUES_-1, applicationId=application_1463520605239_0001, dagName=[550FF659B5B9423299D956DAA0716B68/ED65CA08069B4853A183B14A35F22D41] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-17 21:30:05,407 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F]
2016-05-17 21:30:05,408 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F]
2016-05-17 21:30:05,408 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F]. Starting root input initializers: 1
2016-05-17 21:30:05,409 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:05,409 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 45465605354D45649984A749A5CA4DDE on vertex vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F]
2016-05-17 21:30:05,411 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:05,412 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 45465605354D45649984A749A5CA4DDE asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:05,412 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:05,413 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:05,415 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:05,416 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 45465605354D45649984A749A5CA4DDE on vertex vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F]
2016-05-17 21:30:05,416 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:05,416 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {45465605354D45649984A749A5CA4DDE=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:05,417 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F] parallelism set to 1
2016-05-17 21:30:05,417 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:05,417 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605239_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520605239_0001_1_00, eventCount=1
2016-05-17 21:30:05,417 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F]
2016-05-17 21:30:05,417 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=90847CB6430F4F13A5B66C30FB1359C6, vertexId=vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:05,417 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=90847CB6430F4F13A5B66C30FB1359C6, vertexId=vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F]
2016-05-17 21:30:05,418 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=90847CB6430F4F13A5B66C30FB1359C6, vertexName=C31D863F05CD49889131C51F482CDA1F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:05,419 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=90847CB6430F4F13A5B66C30FB1359C6, vertexId=vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F]
2016-05-17 21:30:05,420 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:05,420 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605239_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C31D863F05CD49889131C51F482CDA1F, vertexId=vertex_1463520605239_0001_1_00, initRequestedTime=1463520605405, initedTime=1463520605419, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:05,420 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F] with distanceFromRoot: 0
2016-05-17 21:30:05,420 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:05,420 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C31D863F05CD49889131C51F482CDA1F
2016-05-17 21:30:05,420 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F]
2016-05-17 21:30:05,420 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:05,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605239_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520605239_0001_1_00, startRequestedTime=1463520605420, startedTime=1463520605420
2016-05-17 21:30:05,421 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:05,421 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:05,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605239_0001_1][Event:TASK_STARTED]: vertexName=C31D863F05CD49889131C51F482CDA1F, taskId=task_1463520605239_0001_1_00_000000, scheduledTime=1463520605421, launchTime=1463520605421
2016-05-17 21:30:05,421 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520605239_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:05,422 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F]
2016-05-17 21:30:05,422 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [550FF659B5B9423299D956DAA0716B68/ED65CA08069B4853A183B14A35F22D41] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1, VertexName: C31D863F05CD49889131C51F482CDA1F, VertexParallelism: 1, TaskAttemptID:attempt_1463520605239_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=45465605354D45649984A749A5CA4DDE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=90847CB6430F4F13A5B66C30FB1359C6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:05,422 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520605239_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:05,422 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:05,423 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:05,423 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:05,423 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:05,424 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520605239_0001_00_000001
2016-05-17 21:30:05,424 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520605239_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:05,424 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520605239_0001, with dag id: dag_1463520605239_0001_1
2016-05-17 21:30:05,426 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520605239_0001_00_000001
2016-05-17 21:30:05,426 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:05,429 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520605239_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:05,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:05,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605239_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520605239_0001_00_000001, launchTime=1463520605429
2016-05-17 21:30:05,432 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520605239_0001_00_000001
2016-05-17 21:30:05,433 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520605239_0001_1_00_000000_0] to container: [container_1463520605239_0001_00_000001]
2016-05-17 21:30:05,433 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520605239_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:05,433 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520605239_0001_00_000001 given task: attempt_1463520605239_0001_1_00_000000_0
2016-05-17 21:30:05,433 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520605239_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520605239_0001_1_00_000000_0
2016-05-17 21:30:05,433 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520605239_0001_1_00_000000_0] started. Is using containerId: [container_1463520605239_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:05 Starting to run new task attempt: attempt_1463520605239_0001_1_00_000000_0
2016-05-17 21:30:05,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605239_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C31D863F05CD49889131C51F482CDA1F, taskAttemptId=attempt_1463520605239_0001_1_00_000000_0, startTime=1463520605433, containerId=container_1463520605239_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520605239_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:05,434 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:05,434 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520605239_0001_1_00_000000_0
2016-05-17 21:30:05,436 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:05,435 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520605239_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:05,436 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520605239_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:05,436 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:05,436 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:05,437 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:05,437 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:05,437 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:05,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [550FF659B5B9423299D956DAA0716B68/ED65CA08069B4853A183B14A35F22D41] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1, VertexName: C31D863F05CD49889131C51F482CDA1F, VertexParallelism: 1, TaskAttemptID:attempt_1463520605239_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=45465605354D45649984A749A5CA4DDE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=90847CB6430F4F13A5B66C30FB1359C6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:05,437 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:05,438 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520605239_0001_1_00_000000_0
2016-05-17 21:30:05,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:05,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:05,439 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520605239_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:05,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=45465605354D45649984A749A5CA4DDE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:05,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=90847CB6430F4F13A5B66C30FB1359C6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:05,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:05,441 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:05,441 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 45465605354D45649984A749A5CA4DDE
2016-05-17 21:30:05,441 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 45465605354D45649984A749A5CA4DDE
2016-05-17 21:30:05,443 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:05,444 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:05,444 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 45465605354D45649984A749A5CA4DDE
2016-05-17 21:30:05,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 45465605354D45649984A749A5CA4DDE
2016-05-17 21:30:05,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:05,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 90847CB6430F4F13A5B66C30FB1359C6
2016-05-17 21:30:05,448 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:05,448 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 90847CB6430F4F13A5B66C30FB1359C6
2016-05-17 21:30:05,454 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C31D863F05CD49889131C51F482CDA1F, ordinal: 0
2016-05-17 21:30:05,454 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C31D863F05CD49889131C51F482CDA1F, mem on start (mb), free: 289, total: 381, max: 672
2016-05-17 21:30:05,454 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:05,454 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:05,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:05,459 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:05,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 90847CB6430F4F13A5B66C30FB1359C6
2016-05-17 21:30:05,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:05,459 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:05,460 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:05,460 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:05,460 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:05,460 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:05,460 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:05,460 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:05,461 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 45465605354D45649984A749A5CA4DDE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:05,461 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 90847CB6430F4F13A5B66C30FB1359C6, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:05,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:05,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 45465605354D45649984A749A5CA4DDE
2016-05-17 21:30:05,461 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:05,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 45465605354D45649984A749A5CA4DDE
2016-05-17 21:30:05,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 45465605354D45649984A749A5CA4DDE being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:05,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:05,463 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:05,463 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:05,463 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520605239_0001_1_00_000000_0
2016-05-17 21:30:05,463 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:05,464 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@468beee0
2016-05-17 21:30:05,465 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@468beee0
2016-05-17 21:30:05,465 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:05,466 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:05,466 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:05,467 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 45465605354D45649984A749A5CA4DDE
2016-05-17 21:30:05,467 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1"], id: 90847CB6430F4F13A5B66C30FB1359C6
2016-05-17 21:30:05,468 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1"]
2016-05-17 21:30:05,468 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:05,468 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:05,469 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C31D863F05CD49889131C51F482CDA1F, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:05,470 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:05,476 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:05,480 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:05,484 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:05,484 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:05,484 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520605239_0001_1_00_000000_0
2016-05-17 21:30:05,485 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520605239_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:05,486 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520605239_0001_1_00_000000_0
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:05,493 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400031744, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463520605469, Process_Duration=16, Process_End_Time=1463520605485, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 21:30:05,494 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C31D863F05CD49889131C51F482CDA1F, mem on close (mb), free: 287, total: 381, max: 672
2016-05-17 21:30:05,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=C31D863F05CD49889131C51F482CDA1F, index=0
2016-05-17 21:30:05,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=C31D863F05CD49889131C51F482CDA1F, sourceVertex=45465605354D45649984A749A5CA4DDE
2016-05-17 21:30:05,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=C31D863F05CD49889131C51F482CDA1F, sourceVertex=90847CB6430F4F13A5B66C30FB1359C6
2016-05-17 21:30:05,495 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520605239_0001_00_000001
2016-05-17 21:30:05,495 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520605239_0001_00_000001
2016-05-17 21:30:05,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:05,496 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605239_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C31D863F05CD49889131C51F482CDA1F, taskAttemptId=attempt_1463520605239_0001_1_00_000000_0, startTime=1463520605433, finishTime=1463520605494, timeTaken=61, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400031744, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520605469, Process_Duration=16, Process_End_Time=1463520605485, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 21:30:05,497 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520605239_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:30:05,497 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520605239_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:30:05,497 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:05,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605239_0001_1][Event:TASK_FINISHED]: vertexName=C31D863F05CD49889131C51F482CDA1F, taskId=task_1463520605239_0001_1_00_000000, startTime=1463520605433, finishTime=1463520605497, timeTaken=64, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
], counters=Counters: 0
2016-05-17 21:30:05,499 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520605239_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:30:05,502 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520605239_0001_00_000001 completed but with errors
2016-05-17 21:30:05,501 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520605239_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:30:05,502 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:05,504 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F] : 1
2016-05-17 21:30:05,504 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F] because task failed: task_1463520605239_0001_1_00_000000
2016-05-17 21:30:05,505 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:30:05,506 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:30:05,506 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F]
2016-05-17 21:30:05,507 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=90847CB6430F4F13A5B66C30FB1359C6, vertexId=vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F]
2016-05-17 21:30:05,511 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:05,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605239_0001_1][Event:VERTEX_FINISHED]: vertexName=C31D863F05CD49889131C51F482CDA1F, vertexId=vertex_1463520605239_0001_1_00, initRequestedTime=1463520605405, initedTime=1463520605419, startRequestedTime=1463520605420, startedTime=1463520605420, finishTime=1463520605510, timeTaken=90, status=FAILED, diagnostics=Task failed, taskId=task_1463520605239_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520605433, firstTasksToStart=[ task_1463520605239_0001_1_00_000000 ], lastTaskFinishTime=1463520605494, lastTasksToFinish=[ task_1463520605239_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:30:05,513 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:30:05,516 INFO container.AMContainerImpl (AMContainerImpl.java:transition(686)) - Container container_1463520605239_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:30:05,516 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:05,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605239_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520605239_0001_00_000001, stoppedTime=1463520605516, exitStatus=1
2016-05-17 21:30:05,517 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520605239_0001_00_000001 transitioned from RUNNING to COMPLETED via event C_COMPLETED
2016-05-17 21:30:05,517 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520605239_0001_1_00_000000_0 failed: true container: container_1463520605239_0001_00_000001 numFailedTAs: 0
2016-05-17 21:30:05,519 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 21:30:05,519 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:30:05,520 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520605239_0001_1
2016-05-17 21:30:05,521 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 90847CB6430F4F13A5B66C30FB1359C6 for vertex: vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F]
2016-05-17 21:30:05,523 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520605239_0001_1, queueSize=0
2016-05-17 21:30:05,523 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:05,524 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605239_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520605239_0001_1, startTime=1463520605404, finishTime=1463520605522, timeTaken=118, status=FAILED, diagnostics=Vertex failed, vertexName=C31D863F05CD49889131C51F482CDA1F, vertexId=vertex_1463520605239_0001_1_00, diagnostics=[Task failed, taskId=task_1463520605239_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520605239_0001_1_00 [C31D863F05CD49889131C51F482CDA1F] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 21:30:05,524 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520605239_0001_1 finished with state: FAILED
2016-05-17 21:30:05,524 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520605239_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:05,525 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520605239_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:30:05 Completed Dag: dag_1463520605239_0001_1
2016-05-17 21:30:05,525 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520605239_0001_1, dagState=FAILED
2016-05-17 21:30:05,525 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520605239_0001_1_post
2016-05-17 21:30:05,526 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:05,526 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:05,529 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...eld'_RESULTS.-1_VALUES_-1, applicationId=application_1463520605239_0001
2016-05-17 21:30:05,533 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:05,533 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:05,533 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:05,533 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:05,533 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:05,534 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:05,534 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:05,536 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:05,536 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:05,536 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:05,536 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:05,536 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:05,537 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:05,541 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/550FF659B5B9423299D956DAA0716B68/ED65CA08069B4853A183B14A35F22D41/.tez/application_1463520605239_0001
2016-05-17 21:30:05,541 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-17 21:30:05,541 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-17 21:30:05,542 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-17 21:30:05,543 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1/_temporary
2016-05-17 21:30:05,543 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.322
2016-05-17 21:30:05,544 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_-1
2016-05-17 21:30:05,566 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:05,567 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:05,605 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-17 21:30:05,608 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-17 21:30:05,608 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:05,609 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:05,609 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_'field2'
2016-05-17 21:30:05,632 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:05,632 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:05,674 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 21:30:05,692 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-05-17 21:30:05,692 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:05,692 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:05,693 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:05,716 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:05,717 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS
2016-05-17 21:30:05,734 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:05,735 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:05,735 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS"]
2016-05-17 21:30:05,735 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:05,735 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:05,735 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:05,736 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-17 21:30:05,736 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:05,755 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:05,759 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:05,760 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:05,760 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:05,769 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE125045D5840578EF1BA98FB9335CC/CC5AE7BDDB2E44E79FAF52FD586850B7/.tez/application_1463520605760_0001 doesn't exist and is created
2016-05-17 21:30:05,786 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:05,786 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:05,786 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE125045D5840578EF1BA98FB9335CC/CC5AE7BDDB2E44E79FAF52FD586850B7/.tez/application_1463520605760_0001
2016-05-17 21:30:05,790 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520605760_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:05,800 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:05,800 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:05,802 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:05,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:05,803 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE125045D5840578EF1BA98FB9335CC/CC5AE7BDDB2E44E79FAF52FD586850B7/.tez/application_1463520605760_0001/localmode-log-dir
2016-05-17 21:30:05,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE125045D5840578EF1BA98FB9335CC/CC5AE7BDDB2E44E79FAF52FD586850B7/.tez/application_1463520605760_0001/localmode-log-dir/history.txt.appattempt_1463520605760_0001_000000, maxErrors=10
2016-05-17 21:30:05,804 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:05,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520605760_0001_000000, appSubmitTime=1463520605789, launchTime=1463520605789
2016-05-17 21:30:05,811 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:05,813 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:42498
2016-05-17 21:30:05,815 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:05,816 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:05,816 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:05,819 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:05,821 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520605760_0001_000000, startTime=1463520605821
2016-05-17 21:30:05,821 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:05,821 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:05,865 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:05,866 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:05,886 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:05,886 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:05,886 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...RESULTS.-1_VALUES_RESULTS, applicationId=application_1463520605760_0001, dagName=[AFE125045D5840578EF1BA98FB9335CC/CC5AE7BDDB2E44E79FAF52FD586850B7] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-17 21:30:05,903 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [AFE125045D5840578EF1BA98FB9335CC/CC5AE7BDDB2E44E79FAF52FD586850B7] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-17 21:30:05,904 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520605760_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE125045D5840578EF1BA98FB9335CC/CC5AE7BDDB2E44E79FAF52FD586850B7/.tez/application_1463520605760_0001/localmode-log-dir/dag_1463520605760_0001_1.dot
2016-05-17 21:30:05,905 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE125045D5840578EF1BA98FB9335CC/CC5AE7BDDB2E44E79FAF52FD586850B7/.tez/application_1463520605760_0001/localmode-log-dir/dag_1463520605760_0001_1-tez-dag.pb.txt
2016-05-17 21:30:05,910 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520605760_0001_1
2016-05-17 21:30:05,910 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:05,910 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [AFE125045D5840578EF1BA98FB9335CC/CC5AE7BDDB2E44E79FAF52FD586850B7] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-17 21:30:05 Running Dag: dag_1463520605760_0001_1
2016-05-17 21:30:05,925 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:05,925 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605760_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520605760_0001_1, submitTime=1463520605903
2016-05-17 21:30:05,926 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B63F3E70E421406B880173E616C24C0F
2016-05-17 21:30:05,926 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:05,927 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:05,927 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605760_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520605760_0001_1, initTime=1463520605926
2016-05-17 21:30:05,927 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520605760_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:05,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605760_0001_1][Event:DAG_STARTED]: dagID=dag_1463520605760_0001_1, startTime=1463520605928
2016-05-17 21:30:05,929 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:05,929 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:05,929 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520605760_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:05,930 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B63F3E70E421406B880173E616C24C0F : {10D3CCB53C6F41FD90BAE4E91839F3D6={InputName=10D3CCB53C6F41FD90BAE4E91839F3D6}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:05,929 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...RESULTS.-1_VALUES_RESULTS, applicationId=application_1463520605760_0001, dagName=[AFE125045D5840578EF1BA98FB9335CC/CC5AE7BDDB2E44E79FAF52FD586850B7] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-17 21:30:05,930 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 10D3CCB53C6F41FD90BAE4E91839F3D6, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:05,930 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F]
2016-05-17 21:30:05,930 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F]
2016-05-17 21:30:05,930 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F]
2016-05-17 21:30:05,930 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F]. Starting root input initializers: 1
2016-05-17 21:30:05,931 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:05,931 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 10D3CCB53C6F41FD90BAE4E91839F3D6 on vertex vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F]
2016-05-17 21:30:05,933 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:05,934 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 10D3CCB53C6F41FD90BAE4E91839F3D6 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:05,934 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:05,935 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:05,937 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:05,938 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 10D3CCB53C6F41FD90BAE4E91839F3D6 on vertex vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F]
2016-05-17 21:30:05,938 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:05,938 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {10D3CCB53C6F41FD90BAE4E91839F3D6=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:05,938 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F] parallelism set to 1
2016-05-17 21:30:05,939 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:05,939 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605760_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520605760_0001_1_00, eventCount=1
2016-05-17 21:30:05,939 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F]
2016-05-17 21:30:05,939 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=58B9044E97B243D0A8963DC52016299F, vertexId=vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:05,939 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=58B9044E97B243D0A8963DC52016299F, vertexId=vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F]
2016-05-17 21:30:05,941 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=58B9044E97B243D0A8963DC52016299F, vertexName=B63F3E70E421406B880173E616C24C0F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:05,942 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=58B9044E97B243D0A8963DC52016299F, vertexId=vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F]
2016-05-17 21:30:05,943 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:05,943 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605760_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B63F3E70E421406B880173E616C24C0F, vertexId=vertex_1463520605760_0001_1_00, initRequestedTime=1463520605930, initedTime=1463520605943, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:05,943 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F] with distanceFromRoot: 0
2016-05-17 21:30:05,944 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:05,944 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B63F3E70E421406B880173E616C24C0F
2016-05-17 21:30:05,945 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F]
2016-05-17 21:30:05,945 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:05,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605760_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520605760_0001_1_00, startRequestedTime=1463520605944, startedTime=1463520605944
2016-05-17 21:30:05,946 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:05,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:05,947 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605760_0001_1][Event:TASK_STARTED]: vertexName=B63F3E70E421406B880173E616C24C0F, taskId=task_1463520605760_0001_1_00_000000, scheduledTime=1463520605946, launchTime=1463520605946
2016-05-17 21:30:05,947 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520605760_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:05,947 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F]
2016-05-17 21:30:05,948 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [AFE125045D5840578EF1BA98FB9335CC/CC5AE7BDDB2E44E79FAF52FD586850B7] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS, VertexName: B63F3E70E421406B880173E616C24C0F, VertexParallelism: 1, TaskAttemptID:attempt_1463520605760_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=10D3CCB53C6F41FD90BAE4E91839F3D6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=58B9044E97B243D0A8963DC52016299F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:05,948 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520605760_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:05,948 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:05,954 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:05,954 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520605760_0001, with dag id: dag_1463520605760_0001_1
2016-05-17 21:30:05,954 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:05,955 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:05,956 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520605760_0001_00_000001
2016-05-17 21:30:05,956 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520605760_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:05,959 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520605760_0001_00_000001
2016-05-17 21:30:05,960 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:05,961 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520605760_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:05,965 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:05,965 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605760_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520605760_0001_00_000001, launchTime=1463520605961
2016-05-17 21:30:05,966 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520605760_0001_00_000001
2016-05-17 21:30:05,969 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520605760_0001_1_00_000000_0] to container: [container_1463520605760_0001_00_000001]
2016-05-17 21:30:05,969 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520605760_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:05,970 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520605760_0001_1_00_000000_0] started. Is using containerId: [container_1463520605760_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:05,971 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:05,971 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605760_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B63F3E70E421406B880173E616C24C0F, taskAttemptId=attempt_1463520605760_0001_1_00_000000_0, startTime=1463520605970, containerId=container_1463520605760_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520605760_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:05,971 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520605760_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:05,971 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520605760_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:05,971 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520605760_0001_00_000001 given task: attempt_1463520605760_0001_1_00_000000_0
2016-05-17 21:30:05,971 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520605760_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520605760_0001_1_00_000000_0
2016-05-17 21:30:05 Starting to run new task attempt: attempt_1463520605760_0001_1_00_000000_0
2016-05-17 21:30:05,972 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520605760_0001_1_00_000000_0
2016-05-17 21:30:05,972 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:05,972 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:05,972 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:05,973 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:05,973 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:05,973 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:05,973 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AFE125045D5840578EF1BA98FB9335CC/CC5AE7BDDB2E44E79FAF52FD586850B7] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS, VertexName: B63F3E70E421406B880173E616C24C0F, VertexParallelism: 1, TaskAttemptID:attempt_1463520605760_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=10D3CCB53C6F41FD90BAE4E91839F3D6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=58B9044E97B243D0A8963DC52016299F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:05,973 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:05,975 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520605760_0001_1_00_000000_0
2016-05-17 21:30:05,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:05,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:05,977 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520605760_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:05,977 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=10D3CCB53C6F41FD90BAE4E91839F3D6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:05,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:05,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 10D3CCB53C6F41FD90BAE4E91839F3D6
2016-05-17 21:30:05,979 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 10D3CCB53C6F41FD90BAE4E91839F3D6
2016-05-17 21:30:05,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=58B9044E97B243D0A8963DC52016299F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:05,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:05,983 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:05,985 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:05,985 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 10D3CCB53C6F41FD90BAE4E91839F3D6
2016-05-17 21:30:05,986 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 10D3CCB53C6F41FD90BAE4E91839F3D6
2016-05-17 21:30:05,987 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:05,987 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 58B9044E97B243D0A8963DC52016299F
2016-05-17 21:30:05,988 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:05,988 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 58B9044E97B243D0A8963DC52016299F
2016-05-17 21:30:05,992 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B63F3E70E421406B880173E616C24C0F, ordinal: 0
2016-05-17 21:30:05,992 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B63F3E70E421406B880173E616C24C0F, mem on start (mb), free: 213, total: 381, max: 672
2016-05-17 21:30:05,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:05,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:05,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:06,002 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:06,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 58B9044E97B243D0A8963DC52016299F
2016-05-17 21:30:06,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:06,002 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:06,003 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:06,003 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:06,003 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:06,003 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:06,003 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:06,003 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:06,004 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 10D3CCB53C6F41FD90BAE4E91839F3D6, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:06,004 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 58B9044E97B243D0A8963DC52016299F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:06,004 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:06,004 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 10D3CCB53C6F41FD90BAE4E91839F3D6 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:06,004 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:06,004 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:06,004 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 10D3CCB53C6F41FD90BAE4E91839F3D6
2016-05-17 21:30:06,004 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:06,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 10D3CCB53C6F41FD90BAE4E91839F3D6
2016-05-17 21:30:06,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:06,005 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520605760_0001_1_00_000000_0
2016-05-17 21:30:06,009 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:06,010 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 10D3CCB53C6F41FD90BAE4E91839F3D6
2016-05-17 21:30:06,010 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS"], id: 58B9044E97B243D0A8963DC52016299F
2016-05-17 21:30:06,011 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS"]
2016-05-17 21:30:06,011 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:06,011 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:06,012 INFO input.MRInput (MRInput.java:checkAndAwaitRecordReaderInitialization(597)) - Awaiting RecordReader initialization
2016-05-17 21:30:06,015 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:06,015 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@601cc787
2016-05-17 21:30:06,017 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@601cc787
2016-05-17 21:30:06,017 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:06,017 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:06,018 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B63F3E70E421406B880173E616C24C0F, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:06,018 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:06,020 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:06,022 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:06,024 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:06,024 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:06,025 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520605760_0001_1_00_000000_0
2016-05-17 21:30:06,025 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520605760_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:06,026 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520605760_0001_1_00_000000_0
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:06,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400031744, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463520606018, Process_Duration=8, Process_End_Time=1463520606026, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 21:30:06,031 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B63F3E70E421406B880173E616C24C0F, mem on close (mb), free: 211, total: 381, max: 672
2016-05-17 21:30:06,031 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605760_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B63F3E70E421406B880173E616C24C0F, taskAttemptId=attempt_1463520605760_0001_1_00_000000_0, startTime=1463520605970, finishTime=1463520606031, timeTaken=61, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400031744, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520606018, Process_Duration=8, Process_End_Time=1463520606026, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 21:30:06,031 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:06,031 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520605760_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:30:06,032 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520605760_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:30:06,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605760_0001_1][Event:TASK_FINISHED]: vertexName=B63F3E70E421406B880173E616C24C0F, taskId=task_1463520605760_0001_1_00_000000, startTime=1463520605970, finishTime=1463520606032, timeTaken=62, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
], counters=Counters: 0
2016-05-17 21:30:06,032 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520605760_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:30:06,033 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F] : 1
2016-05-17 21:30:06,033 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F] because task failed: task_1463520605760_0001_1_00_000000
2016-05-17 21:30:06,033 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:30:06,033 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:30:06,033 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F]
2016-05-17 21:30:06,033 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=58B9044E97B243D0A8963DC52016299F, vertexId=vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F]
2016-05-17 21:30:06,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=B63F3E70E421406B880173E616C24C0F, index=0
2016-05-17 21:30:06,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=B63F3E70E421406B880173E616C24C0F, sourceVertex=10D3CCB53C6F41FD90BAE4E91839F3D6
2016-05-17 21:30:06,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=B63F3E70E421406B880173E616C24C0F, sourceVertex=58B9044E97B243D0A8963DC52016299F
2016-05-17 21:30:06,035 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605760_0001_1][Event:VERTEX_FINISHED]: vertexName=B63F3E70E421406B880173E616C24C0F, vertexId=vertex_1463520605760_0001_1_00, initRequestedTime=1463520605930, initedTime=1463520605943, startRequestedTime=1463520605944, startedTime=1463520605944, finishTime=1463520606035, timeTaken=91, status=FAILED, diagnostics=Task failed, taskId=task_1463520605760_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520605970, firstTasksToStart=[ task_1463520605760_0001_1_00_000000 ], lastTaskFinishTime=1463520606031, lastTasksToFinish=[ task_1463520605760_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:30:06,033 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:06,032 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:06,039 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:06,038 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:30:06,037 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520605760_0001_00_000001
2016-05-17 21:30:06,041 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520605760_0001_00_000001
2016-05-17 21:30:06,041 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 21:30:06,045 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:30:06,045 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520605760_0001_1
2016-05-17 21:30:06,046 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 58B9044E97B243D0A8963DC52016299F for vertex: vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F]
2016-05-17 21:30:06,046 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520605760_0001_1, queueSize=0
2016-05-17 21:30:06,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:06,047 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605760_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520605760_0001_1, startTime=1463520605928, finishTime=1463520606046, timeTaken=118, status=FAILED, diagnostics=Vertex failed, vertexName=B63F3E70E421406B880173E616C24C0F, vertexId=vertex_1463520605760_0001_1_00, diagnostics=[Task failed, taskId=task_1463520605760_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520605760_0001_1_00 [B63F3E70E421406B880173E616C24C0F] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 21:30:06,047 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520605760_0001_1 finished with state: FAILED
2016-05-17 21:30:06,047 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520605760_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:06,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520605760_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520605760_0001_00_000001, stoppedTime=1463520606048, exitStatus=0
2016-05-17 21:30:06,048 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:06,048 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520605760_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:06,049 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520605760_0001_1_00_000000_0 failed: true container: container_1463520605760_0001_00_000001 numFailedTAs: 0
2016-05-17 21:30:06 Completed Dag: dag_1463520605760_0001_1
2016-05-17 21:30:06,050 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520605760_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:30:06,050 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520605760_0001_00_000001 completed but with errors
2016-05-17 21:30:06,050 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520605760_0001_1, dagState=FAILED
2016-05-17 21:30:06,050 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520605760_0001_1_post
2016-05-17 21:30:06,050 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:06,050 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:06,051 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520605760_0001_00_000001
2016-05-17 21:30:06,051 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520605760_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:30:06,052 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520605760_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:06,052 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520605760_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:30:06,053 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...RESULTS.-1_VALUES_RESULTS, applicationId=application_1463520605760_0001
2016-05-17 21:30:06,056 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:06,056 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:06,056 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:06,056 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:06,057 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:06,057 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:06,057 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:06,060 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:06,060 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:06,061 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:06,061 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:06,061 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:06,061 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:06,066 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE125045D5840578EF1BA98FB9335CC/CC5AE7BDDB2E44E79FAF52FD586850B7/.tez/application_1463520605760_0001
2016-05-17 21:30:06,066 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-17 21:30:06,067 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-17 21:30:06,068 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-17 21:30:06,068 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS/_temporary
2016-05-17 21:30:06,073 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.334
2016-05-17 21:30:06,073 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_RESULTS
2016-05-17 21:30:06,101 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:06,101 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:06,140 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 21:30:06,146 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-17 21:30:06,146 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:06,146 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:06,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:06,161 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:06,162 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL
2016-05-17 21:30:06,174 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:06,175 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:06,175 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL"]
2016-05-17 21:30:06,175 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:06,176 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:06,176 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:06,177 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-17 21:30:06,177 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:06,189 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:06,194 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:06,194 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:06,194 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:06,204 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/07ABFA542EB24578BBB235231118DEC9/FDC3BF2C510341E0B70AC8244C5E8897/.tez/application_1463520606194_0001 doesn't exist and is created
2016-05-17 21:30:06,221 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:06,221 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:06,221 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/07ABFA542EB24578BBB235231118DEC9/FDC3BF2C510341E0B70AC8244C5E8897/.tez/application_1463520606194_0001
2016-05-17 21:30:06,225 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520606194_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:06,236 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:06,237 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:06,239 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:06,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:06,240 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/07ABFA542EB24578BBB235231118DEC9/FDC3BF2C510341E0B70AC8244C5E8897/.tez/application_1463520606194_0001/localmode-log-dir
2016-05-17 21:30:06,240 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/07ABFA542EB24578BBB235231118DEC9/FDC3BF2C510341E0B70AC8244C5E8897/.tez/application_1463520606194_0001/localmode-log-dir/history.txt.appattempt_1463520606194_0001_000000, maxErrors=10
2016-05-17 21:30:06,240 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:06,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520606194_0001_000000, appSubmitTime=1463520606224, launchTime=1463520606224
2016-05-17 21:30:06,244 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:06,245 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:06,250 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:06,250 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55841
2016-05-17 21:30:06,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:06,254 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:06,255 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520606194_0001_000000, startTime=1463520606255
2016-05-17 21:30:06,255 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:06,256 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:06,321 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:06,321 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:06,321 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_VALUES_ALL, applicationId=application_1463520606194_0001, dagName=[07ABFA542EB24578BBB235231118DEC9/FDC3BF2C510341E0B70AC8244C5E8897] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-17 21:30:06,337 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [07ABFA542EB24578BBB235231118DEC9/FDC3BF2C510341E0B70AC8244C5E8897] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-17 21:30:06,338 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520606194_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/07ABFA542EB24578BBB235231118DEC9/FDC3BF2C510341E0B70AC8244C5E8897/.tez/application_1463520606194_0001/localmode-log-dir/dag_1463520606194_0001_1.dot
2016-05-17 21:30:06,341 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/07ABFA542EB24578BBB235231118DEC9/FDC3BF2C510341E0B70AC8244C5E8897/.tez/application_1463520606194_0001/localmode-log-dir/dag_1463520606194_0001_1-tez-dag.pb.txt
2016-05-17 21:30:06,346 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520606194_0001_1
2016-05-17 21:30:06,346 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:06,346 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [07ABFA542EB24578BBB235231118DEC9/FDC3BF2C510341E0B70AC8244C5E8897] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-17 21:30:06 Running Dag: dag_1463520606194_0001_1
2016-05-17 21:30:06,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606194_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520606194_0001_1, submitTime=1463520606337
2016-05-17 21:30:06,358 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:06,359 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A611C3B0CF054AEE8505843DA762F028
2016-05-17 21:30:06,359 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:06,360 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606194_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520606194_0001_1, initTime=1463520606358
2016-05-17 21:30:06,360 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:06,361 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520606194_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:06,361 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606194_0001_1][Event:DAG_STARTED]: dagID=dag_1463520606194_0001_1, startTime=1463520606361
2016-05-17 21:30:06,361 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:06,361 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:06,361 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520606194_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:06,362 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A611C3B0CF054AEE8505843DA762F028 : {BAEC2FA0895242F78753C330EBC6C3D5={InputName=BAEC2FA0895242F78753C330EBC6C3D5}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:06,362 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: BAEC2FA0895242F78753C330EBC6C3D5, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:06,362 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028]
2016-05-17 21:30:06,362 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028]
2016-05-17 21:30:06,362 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028]
2016-05-17 21:30:06,363 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028]. Starting root input initializers: 1
2016-05-17 21:30:06,363 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:06,365 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: BAEC2FA0895242F78753C330EBC6C3D5 on vertex vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028]
2016-05-17 21:30:06,365 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_VALUES_ALL, applicationId=application_1463520606194_0001, dagName=[07ABFA542EB24578BBB235231118DEC9/FDC3BF2C510341E0B70AC8244C5E8897] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-17 21:30:06,367 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:06,368 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input BAEC2FA0895242F78753C330EBC6C3D5 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:06,368 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:06,370 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:06,372 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:06,374 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: BAEC2FA0895242F78753C330EBC6C3D5 on vertex vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028]
2016-05-17 21:30:06,375 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:06,375 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {BAEC2FA0895242F78753C330EBC6C3D5=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:06,375 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028] parallelism set to 1
2016-05-17 21:30:06,375 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:06,376 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606194_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520606194_0001_1_00, eventCount=1
2016-05-17 21:30:06,376 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028]
2016-05-17 21:30:06,376 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9E59612A815D4C62BD0F890030636815, vertexId=vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:06,376 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9E59612A815D4C62BD0F890030636815, vertexId=vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028]
2016-05-17 21:30:06,378 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9E59612A815D4C62BD0F890030636815, vertexName=A611C3B0CF054AEE8505843DA762F028, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:06,378 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9E59612A815D4C62BD0F890030636815, vertexId=vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028]
2016-05-17 21:30:06,380 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:06,380 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606194_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A611C3B0CF054AEE8505843DA762F028, vertexId=vertex_1463520606194_0001_1_00, initRequestedTime=1463520606362, initedTime=1463520606380, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:06,380 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028] with distanceFromRoot: 0
2016-05-17 21:30:06,381 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:06,381 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A611C3B0CF054AEE8505843DA762F028
2016-05-17 21:30:06,381 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028]
2016-05-17 21:30:06,382 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:06,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606194_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520606194_0001_1_00, startRequestedTime=1463520606381, startedTime=1463520606381
2016-05-17 21:30:06,382 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:06,383 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:06,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606194_0001_1][Event:TASK_STARTED]: vertexName=A611C3B0CF054AEE8505843DA762F028, taskId=task_1463520606194_0001_1_00_000000, scheduledTime=1463520606383, launchTime=1463520606383
2016-05-17 21:30:06,384 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520606194_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:06,384 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520606194_0001, with dag id: dag_1463520606194_0001_1
2016-05-17 21:30:06,384 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028]
2016-05-17 21:30:06,385 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [07ABFA542EB24578BBB235231118DEC9/FDC3BF2C510341E0B70AC8244C5E8897] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL, VertexName: A611C3B0CF054AEE8505843DA762F028, VertexParallelism: 1, TaskAttemptID:attempt_1463520606194_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BAEC2FA0895242F78753C330EBC6C3D5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9E59612A815D4C62BD0F890030636815, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:06,385 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520606194_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:06,386 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:06,387 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:06,387 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:06,388 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:06,388 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520606194_0001_00_000001
2016-05-17 21:30:06,388 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520606194_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:06,391 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520606194_0001_00_000001
2016-05-17 21:30:06,391 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:06,392 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520606194_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:06,392 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:06,393 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606194_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520606194_0001_00_000001, launchTime=1463520606392
2016-05-17 21:30:06,394 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520606194_0001_00_000001
2016-05-17 21:30:06,394 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520606194_0001_1_00_000000_0] to container: [container_1463520606194_0001_00_000001]
2016-05-17 21:30:06,395 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520606194_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:06,395 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520606194_0001_00_000001 given task: attempt_1463520606194_0001_1_00_000000_0
2016-05-17 21:30:06,395 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520606194_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520606194_0001_1_00_000000_0
2016-05-17 21:30:06,395 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520606194_0001_1_00_000000_0] started. Is using containerId: [container_1463520606194_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:06,396 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606194_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A611C3B0CF054AEE8505843DA762F028, taskAttemptId=attempt_1463520606194_0001_1_00_000000_0, startTime=1463520606395, containerId=container_1463520606194_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520606194_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:06,396 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:06,396 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520606194_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:06 Starting to run new task attempt: attempt_1463520606194_0001_1_00_000000_0
2016-05-17 21:30:06,396 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520606194_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:06,397 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520606194_0001_1_00_000000_0
2016-05-17 21:30:06,397 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:06,397 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:06,397 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:06,398 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:06,398 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:06,398 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:06,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [07ABFA542EB24578BBB235231118DEC9/FDC3BF2C510341E0B70AC8244C5E8897] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL, VertexName: A611C3B0CF054AEE8505843DA762F028, VertexParallelism: 1, TaskAttemptID:attempt_1463520606194_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BAEC2FA0895242F78753C330EBC6C3D5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9E59612A815D4C62BD0F890030636815, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:06,399 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:06,399 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520606194_0001_1_00_000000_0
2016-05-17 21:30:06,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:06,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:06,402 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520606194_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:06,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:06,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BAEC2FA0895242F78753C330EBC6C3D5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:06,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:06,406 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BAEC2FA0895242F78753C330EBC6C3D5
2016-05-17 21:30:06,407 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: BAEC2FA0895242F78753C330EBC6C3D5
2016-05-17 21:30:06,406 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9E59612A815D4C62BD0F890030636815, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:06,406 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:06,408 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:06,410 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:06,410 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: BAEC2FA0895242F78753C330EBC6C3D5
2016-05-17 21:30:06,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BAEC2FA0895242F78753C330EBC6C3D5
2016-05-17 21:30:06,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9E59612A815D4C62BD0F890030636815
2016-05-17 21:30:06,410 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:06,411 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9E59612A815D4C62BD0F890030636815
2016-05-17 21:30:06,419 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A611C3B0CF054AEE8505843DA762F028, ordinal: 0
2016-05-17 21:30:06,420 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A611C3B0CF054AEE8505843DA762F028, mem on start (mb), free: 153, total: 381, max: 672
2016-05-17 21:30:06,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:06,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:06,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:06,423 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:06,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9E59612A815D4C62BD0F890030636815
2016-05-17 21:30:06,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:06,423 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:06,424 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:06,424 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:06,424 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:06,424 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:06,424 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:06,424 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:06,425 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BAEC2FA0895242F78753C330EBC6C3D5, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:06,425 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9E59612A815D4C62BD0F890030636815, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:06,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:06,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BAEC2FA0895242F78753C330EBC6C3D5 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:06,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:06,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:06,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BAEC2FA0895242F78753C330EBC6C3D5
2016-05-17 21:30:06,426 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:06,426 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BAEC2FA0895242F78753C330EBC6C3D5
2016-05-17 21:30:06,426 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:06,426 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520606194_0001_1_00_000000_0
2016-05-17 21:30:06,426 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:06,427 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@49838b82
2016-05-17 21:30:06,433 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:06,433 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: BAEC2FA0895242F78753C330EBC6C3D5
2016-05-17 21:30:06,434 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL"], id: 9E59612A815D4C62BD0F890030636815
2016-05-17 21:30:06,434 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL"]
2016-05-17 21:30:06,434 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:06,435 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:06,434 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@49838b82
2016-05-17 21:30:06,435 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:06,435 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:06,436 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A611C3B0CF054AEE8505843DA762F028, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:06,437 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:06,443 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:06,448 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:06,451 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:06,451 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:06,451 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520606194_0001_1_00_000000_0
2016-05-17 21:30:06,451 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520606194_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:06,452 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520606194_0001_1_00_000000_0
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:06,455 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606194_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A611C3B0CF054AEE8505843DA762F028, taskAttemptId=attempt_1463520606194_0001_1_00_000000_0, startTime=1463520606395, finishTime=1463520606455, timeTaken=60, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400031744, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520606436, Process_Duration=16, Process_End_Time=1463520606452, Read_Duration=1, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 21:30:06,455 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:06,455 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520606194_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:30:06,456 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520606194_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:30:06,456 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606194_0001_1][Event:TASK_FINISHED]: vertexName=A611C3B0CF054AEE8505843DA762F028, taskId=task_1463520606194_0001_1_00_000000, startTime=1463520606395, finishTime=1463520606456, timeTaken=61, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
], counters=Counters: 0
2016-05-17 21:30:06,456 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:06,456 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520606194_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:30:06,457 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028] : 1
2016-05-17 21:30:06,457 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028] because task failed: task_1463520606194_0001_1_00_000000
2016-05-17 21:30:06,457 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:30:06,457 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:30:06,457 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:06,457 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028]
2016-05-17 21:30:06,457 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=9E59612A815D4C62BD0F890030636815, vertexId=vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028]
2016-05-17 21:30:06,460 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:06,460 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606194_0001_1][Event:VERTEX_FINISHED]: vertexName=A611C3B0CF054AEE8505843DA762F028, vertexId=vertex_1463520606194_0001_1_00, initRequestedTime=1463520606362, initedTime=1463520606380, startRequestedTime=1463520606381, startedTime=1463520606381, finishTime=1463520606459, timeTaken=78, status=FAILED, diagnostics=Task failed, taskId=task_1463520606194_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520606395, firstTasksToStart=[ task_1463520606194_0001_1_00_000000 ], lastTaskFinishTime=1463520606455, lastTasksToFinish=[ task_1463520606194_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:30:06,461 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:30:06,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 19 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400031744, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463520606436, Process_Duration=16, Process_End_Time=1463520606452, Read_Duration=1, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 21:30:06,461 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A611C3B0CF054AEE8505843DA762F028, mem on close (mb), free: 150, total: 381, max: 672
2016-05-17 21:30:06,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=A611C3B0CF054AEE8505843DA762F028, index=0
2016-05-17 21:30:06,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=A611C3B0CF054AEE8505843DA762F028, sourceVertex=BAEC2FA0895242F78753C330EBC6C3D5
2016-05-17 21:30:06,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=A611C3B0CF054AEE8505843DA762F028, sourceVertex=9E59612A815D4C62BD0F890030636815
2016-05-17 21:30:06,462 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520606194_0001_00_000001
2016-05-17 21:30:06,462 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:06,462 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520606194_0001_00_000001
2016-05-17 21:30:06,463 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606194_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520606194_0001_00_000001, stoppedTime=1463520606462, exitStatus=0
2016-05-17 21:30:06,463 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520606194_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:06,465 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520606194_0001_1_00_000000_0 failed: true container: container_1463520606194_0001_00_000001 numFailedTAs: 0
2016-05-17 21:30:06,465 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 21:30:06,465 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:30:06,465 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520606194_0001_1
2016-05-17 21:30:06,465 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 9E59612A815D4C62BD0F890030636815 for vertex: vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028]
2016-05-17 21:30:06,466 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520606194_0001_1, queueSize=0
2016-05-17 21:30:06,466 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:06,466 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606194_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520606194_0001_1, startTime=1463520606361, finishTime=1463520606466, timeTaken=105, status=FAILED, diagnostics=Vertex failed, vertexName=A611C3B0CF054AEE8505843DA762F028, vertexId=vertex_1463520606194_0001_1_00, diagnostics=[Task failed, taskId=task_1463520606194_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520606194_0001_1_00 [A611C3B0CF054AEE8505843DA762F028] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 21:30:06,467 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520606194_0001_1 finished with state: FAILED
2016-05-17 21:30:06,467 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520606194_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:06,467 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520606194_0001_00_000001
2016-05-17 21:30:06,467 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520606194_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:30:06,470 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520606194_0001_00_000001 completed but with errors
2016-05-17 21:30:06 Completed Dag: dag_1463520606194_0001_1
2016-05-17 21:30:06,475 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520606194_0001_1, dagState=FAILED
2016-05-17 21:30:06,475 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520606194_0001_1_post
2016-05-17 21:30:06,475 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:06,476 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:06,476 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520606194_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:30:06,476 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520606194_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:30:06,477 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520606194_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:30:06,477 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520606194_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:30:06,483 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.-1_VALUES_ALL, applicationId=application_1463520606194_0001
2016-05-17 21:30:06,489 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:06,489 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:06,490 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:06,490 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:06,490 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:06,490 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:06,490 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:06,491 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:06,491 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:06,491 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:06,491 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:06,492 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:06,492 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:06,496 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/07ABFA542EB24578BBB235231118DEC9/FDC3BF2C510341E0B70AC8244C5E8897/.tez/application_1463520606194_0001
2016-05-17 21:30:06,496 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-17 21:30:06,496 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-17 21:30:06,497 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-17 21:30:06,497 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL/_temporary
2016-05-17 21:30:06,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.323
2016-05-17 21:30:06,498 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_ALL
2016-05-17 21:30:06,519 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:06,520 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:06,527 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:06,527 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:06,577 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.021
2016-05-17 21:30:06,580 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.008
2016-05-17 21:30:06,581 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:06,581 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:06,581 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_ARGS_'field2'
2016-05-17 21:30:06,604 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:06,604 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:06,642 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 21:30:06,648 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-17 21:30:06,649 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:06,649 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:06,649 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:06,663 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:06,663 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS
2016-05-17 21:30:06,674 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:06,674 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:06,674 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS"]
2016-05-17 21:30:06,675 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:06,675 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:06,676 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:06,678 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-17 21:30:06,679 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:06,693 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:06,698 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:06,698 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:06,699 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:06,711 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D2168086DB2642C984D673E50C6C1A5B/B49D3E4D60EB48A59FCF12849C2D4E35/.tez/application_1463520606698_0001 doesn't exist and is created
2016-05-17 21:30:06,730 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:06,730 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:06,731 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D2168086DB2642C984D673E50C6C1A5B/B49D3E4D60EB48A59FCF12849C2D4E35/.tez/application_1463520606698_0001
2016-05-17 21:30:06,735 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520606698_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:06,746 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:06,746 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:06,748 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:06,748 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:06,748 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D2168086DB2642C984D673E50C6C1A5B/B49D3E4D60EB48A59FCF12849C2D4E35/.tez/application_1463520606698_0001/localmode-log-dir
2016-05-17 21:30:06,748 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D2168086DB2642C984D673E50C6C1A5B/B49D3E4D60EB48A59FCF12849C2D4E35/.tez/application_1463520606698_0001/localmode-log-dir/history.txt.appattempt_1463520606698_0001_000000, maxErrors=10
2016-05-17 21:30:06,748 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:06,749 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520606698_0001_000000, appSubmitTime=1463520606735, launchTime=1463520606735
2016-05-17 21:30:06,751 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:06,753 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:06,755 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:06,755 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:06,758 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53047
2016-05-17 21:30:06,759 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:06,761 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520606698_0001_000000, startTime=1463520606761
2016-05-17 21:30:06,761 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:06,761 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:06,832 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:06,833 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:06,833 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_ARGS_RESULTS, applicationId=application_1463520606698_0001, dagName=[D2168086DB2642C984D673E50C6C1A5B/B49D3E4D60EB48A59FCF12849C2D4E35] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-17 21:30:06,848 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D2168086DB2642C984D673E50C6C1A5B/B49D3E4D60EB48A59FCF12849C2D4E35] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-17 21:30:06,849 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520606698_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D2168086DB2642C984D673E50C6C1A5B/B49D3E4D60EB48A59FCF12849C2D4E35/.tez/application_1463520606698_0001/localmode-log-dir/dag_1463520606698_0001_1.dot
2016-05-17 21:30:06,850 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D2168086DB2642C984D673E50C6C1A5B/B49D3E4D60EB48A59FCF12849C2D4E35/.tez/application_1463520606698_0001/localmode-log-dir/dag_1463520606698_0001_1-tez-dag.pb.txt
2016-05-17 21:30:06,855 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520606698_0001_1
2016-05-17 21:30:06,855 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:06,856 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D2168086DB2642C984D673E50C6C1A5B/B49D3E4D60EB48A59FCF12849C2D4E35] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-17 21:30:06 Running Dag: dag_1463520606698_0001_1
2016-05-17 21:30:06,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606698_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520606698_0001_1, submitTime=1463520606848
2016-05-17 21:30:06,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:06,868 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E6F23F767A9B4D2B8A3744DF47630270
2016-05-17 21:30:06,868 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:06,868 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:06,868 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606698_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520606698_0001_1, initTime=1463520606867
2016-05-17 21:30:06,869 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520606698_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:06,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606698_0001_1][Event:DAG_STARTED]: dagID=dag_1463520606698_0001_1, startTime=1463520606869
2016-05-17 21:30:06,870 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:06,870 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520606698_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:06,870 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_ARGS_RESULTS, applicationId=application_1463520606698_0001, dagName=[D2168086DB2642C984D673E50C6C1A5B/B49D3E4D60EB48A59FCF12849C2D4E35] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-17 21:30:06,870 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E6F23F767A9B4D2B8A3744DF47630270 : {5DEF664A8122408A8A36845B74BDA87D={InputName=5DEF664A8122408A8A36845B74BDA87D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:06,870 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 5DEF664A8122408A8A36845B74BDA87D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:06,870 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270]
2016-05-17 21:30:06,871 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270]
2016-05-17 21:30:06,871 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270]
2016-05-17 21:30:06,871 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270]. Starting root input initializers: 1
2016-05-17 21:30:06,871 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:06,872 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:06,872 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 5DEF664A8122408A8A36845B74BDA87D on vertex vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270]
2016-05-17 21:30:06,875 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:06,875 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 5DEF664A8122408A8A36845B74BDA87D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:06,875 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:06,876 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:06,878 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:06,878 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 5DEF664A8122408A8A36845B74BDA87D on vertex vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270]
2016-05-17 21:30:06,878 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:06,879 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {5DEF664A8122408A8A36845B74BDA87D=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:06,879 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270] parallelism set to 1
2016-05-17 21:30:06,879 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:06,879 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606698_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520606698_0001_1_00, eventCount=1
2016-05-17 21:30:06,880 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270]
2016-05-17 21:30:06,880 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=8A9F9269725D4C06AD3037DFEB43FED5, vertexId=vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:06,880 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=8A9F9269725D4C06AD3037DFEB43FED5, vertexId=vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270]
2016-05-17 21:30:06,882 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=8A9F9269725D4C06AD3037DFEB43FED5, vertexName=E6F23F767A9B4D2B8A3744DF47630270, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:06,882 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=8A9F9269725D4C06AD3037DFEB43FED5, vertexId=vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270]
2016-05-17 21:30:06,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606698_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E6F23F767A9B4D2B8A3744DF47630270, vertexId=vertex_1463520606698_0001_1_00, initRequestedTime=1463520606870, initedTime=1463520606882, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:06,883 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270] with distanceFromRoot: 0
2016-05-17 21:30:06,883 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:06,883 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:06,883 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E6F23F767A9B4D2B8A3744DF47630270
2016-05-17 21:30:06,883 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270]
2016-05-17 21:30:06,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606698_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520606698_0001_1_00, startRequestedTime=1463520606883, startedTime=1463520606883
2016-05-17 21:30:06,883 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:06,883 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:06,884 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606698_0001_1][Event:TASK_STARTED]: vertexName=E6F23F767A9B4D2B8A3744DF47630270, taskId=task_1463520606698_0001_1_00_000000, scheduledTime=1463520606884, launchTime=1463520606884
2016-05-17 21:30:06,884 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520606698_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:06,884 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:06,884 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270]
2016-05-17 21:30:06,885 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D2168086DB2642C984D673E50C6C1A5B/B49D3E4D60EB48A59FCF12849C2D4E35] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS, VertexName: E6F23F767A9B4D2B8A3744DF47630270, VertexParallelism: 1, TaskAttemptID:attempt_1463520606698_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5DEF664A8122408A8A36845B74BDA87D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=8A9F9269725D4C06AD3037DFEB43FED5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:06,885 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520606698_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:06,885 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:06,886 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:06,886 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:06,887 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:06,887 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520606698_0001_00_000001
2016-05-17 21:30:06,887 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520606698_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:06,889 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520606698_0001_00_000001
2016-05-17 21:30:06,890 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:06,895 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520606698_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:06,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:06,895 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606698_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520606698_0001_00_000001, launchTime=1463520606895
2016-05-17 21:30:06,897 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520606698_0001_00_000001
2016-05-17 21:30:06,898 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520606698_0001_1_00_000000_0] to container: [container_1463520606698_0001_00_000001]
2016-05-17 21:30:06,898 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520606698_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:06,898 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520606698_0001_00_000001 given task: attempt_1463520606698_0001_1_00_000000_0
2016-05-17 21:30:06,898 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520606698_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520606698_0001_1_00_000000_0
2016-05-17 21:30:06 Starting to run new task attempt: attempt_1463520606698_0001_1_00_000000_0
2016-05-17 21:30:06,899 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520606698_0001_1_00_000000_0] started. Is using containerId: [container_1463520606698_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:06,899 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520606698_0001_1_00_000000_0
2016-05-17 21:30:06,899 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:06,899 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:06,899 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:06,900 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520606698_0001, with dag id: dag_1463520606698_0001_1
2016-05-17 21:30:06,899 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606698_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E6F23F767A9B4D2B8A3744DF47630270, taskAttemptId=attempt_1463520606698_0001_1_00_000000_0, startTime=1463520606899, containerId=container_1463520606698_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520606698_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:06,905 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520606698_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:06,899 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:06,906 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:06,906 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:06,906 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:06,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D2168086DB2642C984D673E50C6C1A5B/B49D3E4D60EB48A59FCF12849C2D4E35] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS, VertexName: E6F23F767A9B4D2B8A3744DF47630270, VertexParallelism: 1, TaskAttemptID:attempt_1463520606698_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5DEF664A8122408A8A36845B74BDA87D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=8A9F9269725D4C06AD3037DFEB43FED5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:06,906 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:06,907 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520606698_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:06,909 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520606698_0001_1_00_000000_0
2016-05-17 21:30:06,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:06,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:06,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=5DEF664A8122408A8A36845B74BDA87D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:06,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=8A9F9269725D4C06AD3037DFEB43FED5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:06,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:06,911 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520606698_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:06,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:06,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 8A9F9269725D4C06AD3037DFEB43FED5
2016-05-17 21:30:06,915 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:06,915 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 8A9F9269725D4C06AD3037DFEB43FED5
2016-05-17 21:30:06,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:06,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 5DEF664A8122408A8A36845B74BDA87D
2016-05-17 21:30:06,916 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 5DEF664A8122408A8A36845B74BDA87D
2016-05-17 21:30:06,915 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:06,918 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:06,918 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 5DEF664A8122408A8A36845B74BDA87D
2016-05-17 21:30:06,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 5DEF664A8122408A8A36845B74BDA87D
2016-05-17 21:30:06,928 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:06,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 8A9F9269725D4C06AD3037DFEB43FED5
2016-05-17 21:30:06,929 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E6F23F767A9B4D2B8A3744DF47630270, ordinal: 0
2016-05-17 21:30:06,929 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E6F23F767A9B4D2B8A3744DF47630270, mem on start (mb), free: 272, total: 379, max: 672
2016-05-17 21:30:06,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:06,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:06,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:06,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:06,931 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:06,931 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:06,931 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:06,931 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:06,931 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:06,932 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:06,932 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:06,932 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 8A9F9269725D4C06AD3037DFEB43FED5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:06,932 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 5DEF664A8122408A8A36845B74BDA87D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:06,932 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:06,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 5DEF664A8122408A8A36845B74BDA87D
2016-05-17 21:30:06,933 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:06,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 5DEF664A8122408A8A36845B74BDA87D
2016-05-17 21:30:06,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 5DEF664A8122408A8A36845B74BDA87D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:06,934 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:06,934 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:06,934 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:06,934 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520606698_0001_1_00_000000_0
2016-05-17 21:30:06,937 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:06,938 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:06,938 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3893055d
2016-05-17 21:30:06,939 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3893055d
2016-05-17 21:30:06,940 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:06,940 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:06,938 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 5DEF664A8122408A8A36845B74BDA87D
2016-05-17 21:30:06,941 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS"], id: 8A9F9269725D4C06AD3037DFEB43FED5
2016-05-17 21:30:06,941 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS"]
2016-05-17 21:30:06,942 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:06,942 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:06,943 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E6F23F767A9B4D2B8A3744DF47630270, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:06,946 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:06,946 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:06,947 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520606698_0001_1_00_000000_0
2016-05-17 21:30:06,947 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520606698_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:06,948 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520606698_0001_1_00_000000_0
2016-05-17 21:30:06,948 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E6F23F767A9B4D2B8A3744DF47630270, mem on close (mb), free: 271, total: 379, max: 672
2016-05-17 21:30:06 Completed running task attempt: attempt_1463520606698_0001_1_00_000000_0
2016-05-17 21:30:06,950 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520606698_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:06,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=397934592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520606943, Process_Duration=5, Process_End_Time=1463520606948, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:06,952 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:06,952 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606698_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E6F23F767A9B4D2B8A3744DF47630270, taskAttemptId=attempt_1463520606698_0001_1_00_000000_0, startTime=1463520606899, finishTime=1463520606951, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=397934592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520606943, Process_Duration=5, Process_End_Time=1463520606948, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:06,952 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520606698_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:06,952 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520606698_0001_1_00_000000_0
2016-05-17 21:30:06,953 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606698_0001_1][Event:TASK_FINISHED]: vertexName=E6F23F767A9B4D2B8A3744DF47630270, taskId=task_1463520606698_0001_1_00_000000, startTime=1463520606899, finishTime=1463520606953, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1463520606698_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=397934592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520606943, Process_Duration=5, Process_End_Time=1463520606948, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:06,953 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520606698_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:06,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:06,953 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:06,953 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270] : 1
2016-05-17 21:30:06,954 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270]
2016-05-17 21:30:06,954 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:06,955 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:06,955 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520606698_0001_00_000001
2016-05-17 21:30:06,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:06,956 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606698_0001_1][Event:VERTEX_FINISHED]: vertexName=E6F23F767A9B4D2B8A3744DF47630270, vertexId=vertex_1463520606698_0001_1_00, initRequestedTime=1463520606870, initedTime=1463520606882, startRequestedTime=1463520606883, startedTime=1463520606883, finishTime=1463520606954, timeTaken=71, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=397934592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520606943, Process_Duration=5, Process_End_Time=1463520606948, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520606899, firstTasksToStart=[ task_1463520606698_0001_1_00_000000 ], lastTaskFinishTime=1463520606951, lastTasksToFinish=[ task_1463520606698_0001_1_00_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520606698_0001_1_00_000000 ], longestDurationTasks=[ task_1463520606698_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:06,956 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:06,956 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520606698_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:06,956 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606698_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520606698_0001_00_000001, stoppedTime=1463520606956, exitStatus=0
2016-05-17 21:30:06,956 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:06,957 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520606698_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:06,957 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:06,957 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520606698_0001_1
2016-05-17 21:30:06,957 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606698_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520606698_0001_1
2016-05-17 21:30:06,957 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 8A9F9269725D4C06AD3037DFEB43FED5 for vertex: vertex_1463520606698_0001_1_00 [E6F23F767A9B4D2B8A3744DF47630270]
2016-05-17 21:30:06,965 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520606698_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:06,966 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520606698_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:06,966 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520606698_0001_00_000001, Exiting
2016-05-17 21:30:06,966 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520606698_0001_00_000001
2016-05-17 21:30:06,967 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520606698_0001_00_000001
2016-05-17 21:30:06,967 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520606698_0001_1, queueSize=0
2016-05-17 21:30:06,968 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:06,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520606698_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520606698_0001_1, startTime=1463520606869, finishTime=1463520606966, timeTaken=97, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=397934592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520606943, Process_Duration=5, Process_End_Time=1463520606948, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:06,969 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520606698_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:06,969 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520606698_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:06,969 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520606698_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:06,969 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520606698_0001_00_000001 completed successfully
2016-05-17 21:30:06 Completed Dag: dag_1463520606698_0001_1
2016-05-17 21:30:06,970 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520606698_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:06,970 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520606698_0001_00_000001
2016-05-17 21:30:06,970 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520606698_0001_1_post
2016-05-17 21:30:06,970 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:06,970 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:06,970 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520606698_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:06,977 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...'_RESULTS.-1_ARGS_RESULTS, applicationId=application_1463520606698_0001
2016-05-17 21:30:06,979 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:06,979 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:06,979 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:06,979 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:06,979 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:06,980 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:06,980 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:06,980 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:06,981 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:06,981 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:06,981 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:06,981 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:06,981 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:06,985 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D2168086DB2642C984D673E50C6C1A5B/B49D3E4D60EB48A59FCF12849C2D4E35/.tez/application_1463520606698_0001
2016-05-17 21:30:06,986 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS/_temporary
2016-05-17 21:30:06,986 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.311
2016-05-17 21:30:07,052 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_ARGS_RESULTS
2016-05-17 21:30:07,073 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:07,074 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:07,100 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 21:30:07,104 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-17 21:30:07,105 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:07,105 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:07,105 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_ARGS_ALL
2016-05-17 21:30:07,125 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:07,126 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:07,163 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:30:07,177 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-17 21:30:07,178 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:07,178 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:07,178 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:07,194 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:07,194 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1
2016-05-17 21:30:07,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:07,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:07,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1"]
2016-05-17 21:30:07,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:07,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:07,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:07,205 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-17 21:30:07,205 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:07,209 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:07,209 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:07,218 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:07,222 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:07,223 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:07,223 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:07,232 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6A14E152605241C88C4FA2C807CA011A/B5C851CB18CC4132BF2A43CF92ECFBD2/.tez/application_1463520607223_0001 doesn't exist and is created
2016-05-17 21:30:07,247 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:07,247 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:07,248 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6A14E152605241C88C4FA2C807CA011A/B5C851CB18CC4132BF2A43CF92ECFBD2/.tez/application_1463520607223_0001
2016-05-17 21:30:07,250 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520607223_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:07,260 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:07,261 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:07,263 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:07,263 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:07,264 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6A14E152605241C88C4FA2C807CA011A/B5C851CB18CC4132BF2A43CF92ECFBD2/.tez/application_1463520607223_0001/localmode-log-dir
2016-05-17 21:30:07,264 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6A14E152605241C88C4FA2C807CA011A/B5C851CB18CC4132BF2A43CF92ECFBD2/.tez/application_1463520607223_0001/localmode-log-dir/history.txt.appattempt_1463520607223_0001_000000, maxErrors=10
2016-05-17 21:30:07,264 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:07,265 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520607223_0001_000000, appSubmitTime=1463520607250, launchTime=1463520607250
2016-05-17 21:30:07,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:07,273 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:07,274 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55619
2016-05-17 21:30:07,274 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:07,276 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:07,277 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:07,280 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520607223_0001_000000, startTime=1463520607280
2016-05-17 21:30:07,280 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:07,281 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:07,347 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:07,348 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:07,348 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_'field2'_-1, applicationId=application_1463520607223_0001, dagName=[6A14E152605241C88C4FA2C807CA011A/B5C851CB18CC4132BF2A43CF92ECFBD2] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-17 21:30:07,364 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [6A14E152605241C88C4FA2C807CA011A/B5C851CB18CC4132BF2A43CF92ECFBD2] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-17 21:30:07,367 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520607223_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6A14E152605241C88C4FA2C807CA011A/B5C851CB18CC4132BF2A43CF92ECFBD2/.tez/application_1463520607223_0001/localmode-log-dir/dag_1463520607223_0001_1.dot
2016-05-17 21:30:07,368 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6A14E152605241C88C4FA2C807CA011A/B5C851CB18CC4132BF2A43CF92ECFBD2/.tez/application_1463520607223_0001/localmode-log-dir/dag_1463520607223_0001_1-tez-dag.pb.txt
2016-05-17 21:30:07,373 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520607223_0001_1
2016-05-17 21:30:07,374 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:07,374 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [6A14E152605241C88C4FA2C807CA011A/B5C851CB18CC4132BF2A43CF92ECFBD2] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-17 21:30:07 Running Dag: dag_1463520607223_0001_1
2016-05-17 21:30:07,389 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607223_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520607223_0001_1, submitTime=1463520607365
2016-05-17 21:30:07,389 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:07,389 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E4A2B0EB5C2549E888684ACA48367E26
2016-05-17 21:30:07,389 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:07,390 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607223_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520607223_0001_1, initTime=1463520607389
2016-05-17 21:30:07,390 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:07,391 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520607223_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:07,391 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607223_0001_1][Event:DAG_STARTED]: dagID=dag_1463520607223_0001_1, startTime=1463520607391
2016-05-17 21:30:07,391 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:07,391 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:07,391 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520607223_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:07,392 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E4A2B0EB5C2549E888684ACA48367E26 : {8C529B73C1FD42A29FB97E8D8FC4F6FA={InputName=8C529B73C1FD42A29FB97E8D8FC4F6FA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:07,392 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 8C529B73C1FD42A29FB97E8D8FC4F6FA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:07,392 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26]
2016-05-17 21:30:07,392 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26]
2016-05-17 21:30:07,392 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26]
2016-05-17 21:30:07,392 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26]. Starting root input initializers: 1
2016-05-17 21:30:07,393 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:07,393 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 8C529B73C1FD42A29FB97E8D8FC4F6FA on vertex vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26]
2016-05-17 21:30:07,393 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_'field2'_-1, applicationId=application_1463520607223_0001, dagName=[6A14E152605241C88C4FA2C807CA011A/B5C851CB18CC4132BF2A43CF92ECFBD2] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-17 21:30:07,395 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:07,395 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 8C529B73C1FD42A29FB97E8D8FC4F6FA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:07,395 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:07,395 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:07,397 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:07,402 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 8C529B73C1FD42A29FB97E8D8FC4F6FA on vertex vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26]
2016-05-17 21:30:07,402 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:07,403 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {8C529B73C1FD42A29FB97E8D8FC4F6FA=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:07,403 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26] parallelism set to 1
2016-05-17 21:30:07,403 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:07,403 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607223_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520607223_0001_1_00, eventCount=1
2016-05-17 21:30:07,403 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26]
2016-05-17 21:30:07,404 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=8D7D59FC31954AE380E4412D94B8F1EE, vertexId=vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:07,404 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=8D7D59FC31954AE380E4412D94B8F1EE, vertexId=vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26]
2016-05-17 21:30:07,413 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520607223_0001, with dag id: dag_1463520607223_0001_1
2016-05-17 21:30:07,416 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=8D7D59FC31954AE380E4412D94B8F1EE, vertexName=E4A2B0EB5C2549E888684ACA48367E26, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:07,417 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=8D7D59FC31954AE380E4412D94B8F1EE, vertexId=vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26]
2016-05-17 21:30:07,418 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607223_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E4A2B0EB5C2549E888684ACA48367E26, vertexId=vertex_1463520607223_0001_1_00, initRequestedTime=1463520607392, initedTime=1463520607418, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:07,418 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26] with distanceFromRoot: 0
2016-05-17 21:30:07,418 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:07,418 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:07,419 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E4A2B0EB5C2549E888684ACA48367E26
2016-05-17 21:30:07,419 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26]
2016-05-17 21:30:07,419 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607223_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520607223_0001_1_00, startRequestedTime=1463520607419, startedTime=1463520607419
2016-05-17 21:30:07,419 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:07,420 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:07,420 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:07,420 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607223_0001_1][Event:TASK_STARTED]: vertexName=E4A2B0EB5C2549E888684ACA48367E26, taskId=task_1463520607223_0001_1_00_000000, scheduledTime=1463520607420, launchTime=1463520607420
2016-05-17 21:30:07,422 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520607223_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:07,422 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26]
2016-05-17 21:30:07,423 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6A14E152605241C88C4FA2C807CA011A/B5C851CB18CC4132BF2A43CF92ECFBD2] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1, VertexName: E4A2B0EB5C2549E888684ACA48367E26, VertexParallelism: 1, TaskAttemptID:attempt_1463520607223_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8C529B73C1FD42A29FB97E8D8FC4F6FA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=8D7D59FC31954AE380E4412D94B8F1EE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:07,423 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520607223_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:07,424 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:07,425 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:07,426 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:07,426 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:07,426 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520607223_0001_00_000001
2016-05-17 21:30:07,427 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520607223_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:07,428 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520607223_0001_00_000001
2016-05-17 21:30:07,429 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:07,430 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520607223_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:07,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:07,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607223_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520607223_0001_00_000001, launchTime=1463520607430
2016-05-17 21:30:07,431 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520607223_0001_00_000001
2016-05-17 21:30:07,431 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520607223_0001_1_00_000000_0] to container: [container_1463520607223_0001_00_000001]
2016-05-17 21:30:07,432 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520607223_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:07,432 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520607223_0001_1_00_000000_0] started. Is using containerId: [container_1463520607223_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:07,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607223_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E4A2B0EB5C2549E888684ACA48367E26, taskAttemptId=attempt_1463520607223_0001_1_00_000000_0, startTime=1463520607432, containerId=container_1463520607223_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520607223_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:07,433 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:07,433 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520607223_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:07,433 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520607223_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:07,433 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520607223_0001_00_000001 given task: attempt_1463520607223_0001_1_00_000000_0
2016-05-17 21:30:07,433 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520607223_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520607223_0001_1_00_000000_0
2016-05-17 21:30:07 Starting to run new task attempt: attempt_1463520607223_0001_1_00_000000_0
2016-05-17 21:30:07,434 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520607223_0001_1_00_000000_0
2016-05-17 21:30:07,434 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:07,434 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:07,434 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:07,436 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:07,436 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:07,437 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:07,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6A14E152605241C88C4FA2C807CA011A/B5C851CB18CC4132BF2A43CF92ECFBD2] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1, VertexName: E4A2B0EB5C2549E888684ACA48367E26, VertexParallelism: 1, TaskAttemptID:attempt_1463520607223_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8C529B73C1FD42A29FB97E8D8FC4F6FA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=8D7D59FC31954AE380E4412D94B8F1EE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:07,437 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:07,438 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520607223_0001_1_00_000000_0
2016-05-17 21:30:07,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:07,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:07,438 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520607223_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:07,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=8C529B73C1FD42A29FB97E8D8FC4F6FA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:07,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=8D7D59FC31954AE380E4412D94B8F1EE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:07,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:07,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:07,441 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 8C529B73C1FD42A29FB97E8D8FC4F6FA
2016-05-17 21:30:07,441 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 8C529B73C1FD42A29FB97E8D8FC4F6FA
2016-05-17 21:30:07,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:07,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 8D7D59FC31954AE380E4412D94B8F1EE
2016-05-17 21:30:07,443 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:07,444 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 8D7D59FC31954AE380E4412D94B8F1EE
2016-05-17 21:30:07,444 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:07,445 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 8C529B73C1FD42A29FB97E8D8FC4F6FA
2016-05-17 21:30:07,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 8C529B73C1FD42A29FB97E8D8FC4F6FA
2016-05-17 21:30:07,450 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:07,456 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:07,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 8D7D59FC31954AE380E4412D94B8F1EE
2016-05-17 21:30:07,464 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E4A2B0EB5C2549E888684ACA48367E26, ordinal: 0
2016-05-17 21:30:07,464 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4A2B0EB5C2549E888684ACA48367E26, mem on start (mb), free: 182, total: 379, max: 672
2016-05-17 21:30:07,464 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:07,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:07,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:07,466 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:07,466 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:07,467 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:07,467 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:07,467 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:07,468 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:07,468 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:07,469 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:07,469 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 8C529B73C1FD42A29FB97E8D8FC4F6FA, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:07,469 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 8D7D59FC31954AE380E4412D94B8F1EE, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:07,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:07,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 8C529B73C1FD42A29FB97E8D8FC4F6FA
2016-05-17 21:30:07,470 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:07,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 8C529B73C1FD42A29FB97E8D8FC4F6FA
2016-05-17 21:30:07,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 8C529B73C1FD42A29FB97E8D8FC4F6FA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:07,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:07,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:07,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:07,472 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520607223_0001_1_00_000000_0
2016-05-17 21:30:07,472 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:07,473 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7e819c62
2016-05-17 21:30:07,474 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7e819c62
2016-05-17 21:30:07,474 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:07,474 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:07,476 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:07,477 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 8C529B73C1FD42A29FB97E8D8FC4F6FA
2016-05-17 21:30:07,477 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1"], id: 8D7D59FC31954AE380E4412D94B8F1EE
2016-05-17 21:30:07,477 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1"]
2016-05-17 21:30:07,478 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:07,478 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:07,478 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E4A2B0EB5C2549E888684ACA48367E26, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:07,481 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:07,481 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:07,482 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520607223_0001_1_00_000000_0
2016-05-17 21:30:07,482 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520607223_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:07,483 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520607223_0001_1_00_000000_0
2016-05-17 21:30:07,483 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4A2B0EB5C2549E888684ACA48367E26, mem on close (mb), free: 181, total: 379, max: 672
2016-05-17 21:30:07 Completed running task attempt: attempt_1463520607223_0001_1_00_000000_0
2016-05-17 21:30:07,484 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520607223_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:07,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=397934592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520607478, Process_Duration=5, Process_End_Time=1463520607483, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:07,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607223_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E4A2B0EB5C2549E888684ACA48367E26, taskAttemptId=attempt_1463520607223_0001_1_00_000000_0, startTime=1463520607432, finishTime=1463520607485, timeTaken=53, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=397934592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520607478, Process_Duration=5, Process_End_Time=1463520607483, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:07,486 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:07,486 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520607223_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:07,487 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:07,487 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:07,487 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520607223_0001_00_000001
2016-05-17 21:30:07,487 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:07,487 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520607223_0001_1_00_000000_0
2016-05-17 21:30:07,488 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:07,489 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607223_0001_1][Event:TASK_FINISHED]: vertexName=E4A2B0EB5C2549E888684ACA48367E26, taskId=task_1463520607223_0001_1_00_000000, startTime=1463520607432, finishTime=1463520607488, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1463520607223_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=397934592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520607478, Process_Duration=5, Process_End_Time=1463520607483, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:07,489 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520607223_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:07,489 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26] : 1
2016-05-17 21:30:07,489 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26]
2016-05-17 21:30:07,490 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:07,491 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607223_0001_1][Event:VERTEX_FINISHED]: vertexName=E4A2B0EB5C2549E888684ACA48367E26, vertexId=vertex_1463520607223_0001_1_00, initRequestedTime=1463520607392, initedTime=1463520607418, startRequestedTime=1463520607419, startedTime=1463520607419, finishTime=1463520607489, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=397934592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520607478, Process_Duration=5, Process_End_Time=1463520607483, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520607432, firstTasksToStart=[ task_1463520607223_0001_1_00_000000 ], lastTaskFinishTime=1463520607485, lastTasksToFinish=[ task_1463520607223_0001_1_00_000000 ], minTaskDuration=53, maxTaskDuration=53, avgTaskDuration=53.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520607223_0001_1_00_000000 ], longestDurationTasks=[ task_1463520607223_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:07,491 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:07,492 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520607223_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:07,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607223_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520607223_0001_00_000001, stoppedTime=1463520607492, exitStatus=0
2016-05-17 21:30:07,492 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:07,492 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520607223_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:07,493 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:07,493 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520607223_0001_1
2016-05-17 21:30:07,495 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607223_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520607223_0001_1
2016-05-17 21:30:07,496 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 8D7D59FC31954AE380E4412D94B8F1EE for vertex: vertex_1463520607223_0001_1_00 [E4A2B0EB5C2549E888684ACA48367E26]
2016-05-17 21:30:07,499 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520607223_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:07,500 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520607223_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:07,500 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520607223_0001_00_000001, Exiting
2016-05-17 21:30:07,500 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520607223_0001_00_000001
2016-05-17 21:30:07,500 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520607223_0001_00_000001
2016-05-17 21:30:07,502 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520607223_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:07,502 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520607223_0001_00_000001 completed successfully
2016-05-17 21:30:07,506 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520607223_0001_1, queueSize=0
2016-05-17 21:30:07,507 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:07,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607223_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520607223_0001_1, startTime=1463520607391, finishTime=1463520607505, timeTaken=114, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=397934592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520607478, Process_Duration=5, Process_End_Time=1463520607483, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:07,507 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520607223_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:07,507 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520607223_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:07,507 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520607223_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:07 Completed Dag: dag_1463520607223_0001_1
2016-05-17 21:30:07,508 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520607223_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:07,508 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520607223_0001_1_post
2016-05-17 21:30:07,508 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:07,508 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:07,508 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520607223_0001_00_000001
2016-05-17 21:30:07,514 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.0_'field2'_-1, applicationId=application_1463520607223_0001
2016-05-17 21:30:07,516 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:07,516 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:07,516 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:07,516 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:07,516 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:07,516 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:07,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:07,518 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:07,518 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:07,518 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:07,518 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:07,519 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:07,519 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:07,524 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6A14E152605241C88C4FA2C807CA011A/B5C851CB18CC4132BF2A43CF92ECFBD2/.tez/application_1463520607223_0001
2016-05-17 21:30:07,524 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1/_temporary
2016-05-17 21:30:07,525 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.321
2016-05-17 21:30:07,589 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_-1
2016-05-17 21:30:07,619 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:07,619 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:07,671 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 21:30:07,679 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-17 21:30:07,680 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:07,680 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:07,680 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:07,694 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:07,695 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'
2016-05-17 21:30:07,706 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:07,706 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:07,707 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'"]
2016-05-17 21:30:07,707 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:07,707 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:07,708 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:07,708 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...SULTS.0_'field2'_'field2'
2016-05-17 21:30:07,708 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:07,720 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:07,726 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:07,727 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:07,727 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:07,737 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2C61DA7E551A4A348E9E1D2FB3173324/CC6E1976B0DD4024A2BBBDB6ACA9CDFC/.tez/application_1463520607727_0001 doesn't exist and is created
2016-05-17 21:30:07,745 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:07,745 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:07,752 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:07,752 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:07,753 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2C61DA7E551A4A348E9E1D2FB3173324/CC6E1976B0DD4024A2BBBDB6ACA9CDFC/.tez/application_1463520607727_0001
2016-05-17 21:30:07,756 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520607727_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:07,768 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:07,768 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:07,770 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:07,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:07,771 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2C61DA7E551A4A348E9E1D2FB3173324/CC6E1976B0DD4024A2BBBDB6ACA9CDFC/.tez/application_1463520607727_0001/localmode-log-dir
2016-05-17 21:30:07,771 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2C61DA7E551A4A348E9E1D2FB3173324/CC6E1976B0DD4024A2BBBDB6ACA9CDFC/.tez/application_1463520607727_0001/localmode-log-dir/history.txt.appattempt_1463520607727_0001_000000, maxErrors=10
2016-05-17 21:30:07,771 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:07,773 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520607727_0001_000000, appSubmitTime=1463520607755, launchTime=1463520607755
2016-05-17 21:30:07,778 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:07,778 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:07,779 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:07,779 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:07,798 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41648
2016-05-17 21:30:07,800 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:07,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520607727_0001_000000, startTime=1463520607802
2016-05-17 21:30:07,803 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:07,803 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:07,853 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:07,853 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:07,853 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...SULTS.0_'field2'_'field2', applicationId=application_1463520607727_0001, dagName=[2C61DA7E551A4A348E9E1D2FB3173324/CC6E1976B0DD4024A2BBBDB6ACA9CDFC] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-17 21:30:07,868 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2C61DA7E551A4A348E9E1D2FB3173324/CC6E1976B0DD4024A2BBBDB6ACA9CDFC] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-17 21:30:07,869 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520607727_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2C61DA7E551A4A348E9E1D2FB3173324/CC6E1976B0DD4024A2BBBDB6ACA9CDFC/.tez/application_1463520607727_0001/localmode-log-dir/dag_1463520607727_0001_1.dot
2016-05-17 21:30:07,869 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2C61DA7E551A4A348E9E1D2FB3173324/CC6E1976B0DD4024A2BBBDB6ACA9CDFC/.tez/application_1463520607727_0001/localmode-log-dir/dag_1463520607727_0001_1-tez-dag.pb.txt
2016-05-17 21:30:07,875 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520607727_0001_1
2016-05-17 21:30:07,875 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:07,875 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2C61DA7E551A4A348E9E1D2FB3173324/CC6E1976B0DD4024A2BBBDB6ACA9CDFC] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-17 21:30:07 Running Dag: dag_1463520607727_0001_1
2016-05-17 21:30:07,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607727_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520607727_0001_1, submitTime=1463520607868
2016-05-17 21:30:07,887 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:07,887 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A3D32269F5D84EF0B87F047B5D3D4E71
2016-05-17 21:30:07,887 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:07,889 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:07,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607727_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520607727_0001_1, initTime=1463520607887
2016-05-17 21:30:07,889 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520607727_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:07,890 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607727_0001_1][Event:DAG_STARTED]: dagID=dag_1463520607727_0001_1, startTime=1463520607890
2016-05-17 21:30:07,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:07,890 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:07,891 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520607727_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:07,891 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A3D32269F5D84EF0B87F047B5D3D4E71 : {64FF28B52B2A439FA654902C93885112={InputName=64FF28B52B2A439FA654902C93885112}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:07,891 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...SULTS.0_'field2'_'field2', applicationId=application_1463520607727_0001, dagName=[2C61DA7E551A4A348E9E1D2FB3173324/CC6E1976B0DD4024A2BBBDB6ACA9CDFC] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-17 21:30:07,891 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 64FF28B52B2A439FA654902C93885112, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:07,891 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71]
2016-05-17 21:30:07,891 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71]
2016-05-17 21:30:07,891 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71]
2016-05-17 21:30:07,892 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71]. Starting root input initializers: 1
2016-05-17 21:30:07,892 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:07,893 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 64FF28B52B2A439FA654902C93885112 on vertex vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71]
2016-05-17 21:30:07,894 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:07,894 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 64FF28B52B2A439FA654902C93885112 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:07,895 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:07,895 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:07,896 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:07,896 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 64FF28B52B2A439FA654902C93885112 on vertex vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71]
2016-05-17 21:30:07,897 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:07,897 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {64FF28B52B2A439FA654902C93885112=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:07,897 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71] parallelism set to 1
2016-05-17 21:30:07,897 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:07,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607727_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520607727_0001_1_00, eventCount=1
2016-05-17 21:30:07,898 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71]
2016-05-17 21:30:07,898 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=48F4F8544E3145C6885FEF9F7F951883, vertexId=vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:07,898 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=48F4F8544E3145C6885FEF9F7F951883, vertexId=vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71]
2016-05-17 21:30:07,899 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=48F4F8544E3145C6885FEF9F7F951883, vertexName=A3D32269F5D84EF0B87F047B5D3D4E71, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:07,900 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=48F4F8544E3145C6885FEF9F7F951883, vertexId=vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71]
2016-05-17 21:30:07,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607727_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A3D32269F5D84EF0B87F047B5D3D4E71, vertexId=vertex_1463520607727_0001_1_00, initRequestedTime=1463520607891, initedTime=1463520607900, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:07,901 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71] with distanceFromRoot: 0
2016-05-17 21:30:07,900 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:07,901 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:07,901 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A3D32269F5D84EF0B87F047B5D3D4E71
2016-05-17 21:30:07,901 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71]
2016-05-17 21:30:07,902 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607727_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520607727_0001_1_00, startRequestedTime=1463520607901, startedTime=1463520607901
2016-05-17 21:30:07,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:07,902 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:07,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:07,902 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607727_0001_1][Event:TASK_STARTED]: vertexName=A3D32269F5D84EF0B87F047B5D3D4E71, taskId=task_1463520607727_0001_1_00_000000, scheduledTime=1463520607902, launchTime=1463520607902
2016-05-17 21:30:07,903 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520607727_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:07,903 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71]
2016-05-17 21:30:07,903 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2C61DA7E551A4A348E9E1D2FB3173324/CC6E1976B0DD4024A2BBBDB6ACA9CDFC] every.every/(1/1) ...SULTS.0_'field2'_'field2', VertexName: A3D32269F5D84EF0B87F047B5D3D4E71, VertexParallelism: 1, TaskAttemptID:attempt_1463520607727_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=64FF28B52B2A439FA654902C93885112, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=48F4F8544E3145C6885FEF9F7F951883, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:07,903 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520607727_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:07,904 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:07,904 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:07,905 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:07,905 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:07,905 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520607727_0001_00_000001
2016-05-17 21:30:07,905 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520607727_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:07,907 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520607727_0001_00_000001
2016-05-17 21:30:07,907 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:07,908 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520607727_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:07,908 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607727_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520607727_0001_00_000001, launchTime=1463520607908
2016-05-17 21:30:07,908 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:07,909 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520607727_0001_00_000001
2016-05-17 21:30:07,911 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520607727_0001_1_00_000000_0] to container: [container_1463520607727_0001_00_000001]
2016-05-17 21:30:07,912 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520607727_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:07,914 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520607727_0001_00_000001 given task: attempt_1463520607727_0001_1_00_000000_0
2016-05-17 21:30:07,914 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520607727_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520607727_0001_1_00_000000_0
2016-05-17 21:30:07,914 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520607727_0001_1_00_000000_0] started. Is using containerId: [container_1463520607727_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:07 Starting to run new task attempt: attempt_1463520607727_0001_1_00_000000_0
2016-05-17 21:30:07,915 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520607727_0001_1_00_000000_0
2016-05-17 21:30:07,915 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:07,915 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:07,915 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:07,915 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:07,915 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:07,915 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:07,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2C61DA7E551A4A348E9E1D2FB3173324/CC6E1976B0DD4024A2BBBDB6ACA9CDFC] every.every/(1/1) ...SULTS.0_'field2'_'field2', VertexName: A3D32269F5D84EF0B87F047B5D3D4E71, VertexParallelism: 1, TaskAttemptID:attempt_1463520607727_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=64FF28B52B2A439FA654902C93885112, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=48F4F8544E3145C6885FEF9F7F951883, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:07,916 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:07,918 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:07,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607727_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A3D32269F5D84EF0B87F047B5D3D4E71, taskAttemptId=attempt_1463520607727_0001_1_00_000000_0, startTime=1463520607914, containerId=container_1463520607727_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520607727_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:07,918 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520607727_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:07,916 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520607727_0001_1_00_000000_0
2016-05-17 21:30:07,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:07,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:07,918 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520607727_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:07,920 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520607727_0001, with dag id: dag_1463520607727_0001_1
2016-05-17 21:30:07,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=64FF28B52B2A439FA654902C93885112, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:07,921 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:07,921 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 64FF28B52B2A439FA654902C93885112
2016-05-17 21:30:07,922 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 64FF28B52B2A439FA654902C93885112
2016-05-17 21:30:07,921 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:07,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=48F4F8544E3145C6885FEF9F7F951883, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:07,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:07,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 48F4F8544E3145C6885FEF9F7F951883
2016-05-17 21:30:07,924 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:07,920 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520607727_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:07,925 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:07,924 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 48F4F8544E3145C6885FEF9F7F951883
2016-05-17 21:30:07,926 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:07,927 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 64FF28B52B2A439FA654902C93885112
2016-05-17 21:30:07,927 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 64FF28B52B2A439FA654902C93885112
2016-05-17 21:30:07,935 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A3D32269F5D84EF0B87F047B5D3D4E71, ordinal: 0
2016-05-17 21:30:07,935 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A3D32269F5D84EF0B87F047B5D3D4E71, mem on start (mb), free: 305, total: 380, max: 672
2016-05-17 21:30:07,935 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:07,935 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:07,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:07,936 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:07,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 48F4F8544E3145C6885FEF9F7F951883
2016-05-17 21:30:07,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:07,937 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:07,937 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:07,937 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:07,937 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:07,937 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:07,937 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:07,938 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:07,938 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 64FF28B52B2A439FA654902C93885112, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:07,938 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 48F4F8544E3145C6885FEF9F7F951883, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:07,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:07,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 64FF28B52B2A439FA654902C93885112
2016-05-17 21:30:07,938 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:07,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 64FF28B52B2A439FA654902C93885112
2016-05-17 21:30:07,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 64FF28B52B2A439FA654902C93885112 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:07,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:07,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:07,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:07,940 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520607727_0001_1_00_000000_0
2016-05-17 21:30:07,942 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:07,943 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:07,943 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 64FF28B52B2A439FA654902C93885112
2016-05-17 21:30:07,943 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'"], id: 48F4F8544E3145C6885FEF9F7F951883
2016-05-17 21:30:07,943 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'"]
2016-05-17 21:30:07,944 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:07,944 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:07,944 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7ff0c6a8
2016-05-17 21:30:07,945 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7ff0c6a8
2016-05-17 21:30:07,945 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:07,945 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:07,946 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A3D32269F5D84EF0B87F047B5D3D4E71, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:07,949 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:07,949 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:07,950 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520607727_0001_1_00_000000_0
2016-05-17 21:30:07,950 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520607727_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:07,951 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520607727_0001_1_00_000000_0
2016-05-17 21:30:07,951 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A3D32269F5D84EF0B87F047B5D3D4E71, mem on close (mb), free: 303, total: 380, max: 672
2016-05-17 21:30:07 Completed running task attempt: attempt_1463520607727_0001_1_00_000000_0
2016-05-17 21:30:07,952 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520607727_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:07,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:07,953 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607727_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A3D32269F5D84EF0B87F047B5D3D4E71, taskAttemptId=attempt_1463520607727_0001_1_00_000000_0, startTime=1463520607914, finishTime=1463520607953, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=398983168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520607946, Process_Duration=5, Process_End_Time=1463520607951, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:07,954 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520607727_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:07,954 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:07,954 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520607727_0001_1_00_000000_0
2016-05-17 21:30:07,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:07,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607727_0001_1][Event:TASK_FINISHED]: vertexName=A3D32269F5D84EF0B87F047B5D3D4E71, taskId=task_1463520607727_0001_1_00_000000, startTime=1463520607914, finishTime=1463520607954, timeTaken=40, status=SUCCEEDED, successfulAttemptID=attempt_1463520607727_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=398983168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520607946, Process_Duration=5, Process_End_Time=1463520607951, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:07,955 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520607727_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:07,955 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71] : 1
2016-05-17 21:30:07,955 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71]
2016-05-17 21:30:07,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=398983168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520607946, Process_Duration=5, Process_End_Time=1463520607951, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:07,956 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:07,956 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:07,956 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:07,957 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520607727_0001_00_000001
2016-05-17 21:30:07,958 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607727_0001_1][Event:VERTEX_FINISHED]: vertexName=A3D32269F5D84EF0B87F047B5D3D4E71, vertexId=vertex_1463520607727_0001_1_00, initRequestedTime=1463520607891, initedTime=1463520607900, startRequestedTime=1463520607901, startedTime=1463520607901, finishTime=1463520607955, timeTaken=54, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=398983168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520607946, Process_Duration=5, Process_End_Time=1463520607951, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520607914, firstTasksToStart=[ task_1463520607727_0001_1_00_000000 ], lastTaskFinishTime=1463520607953, lastTasksToFinish=[ task_1463520607727_0001_1_00_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520607727_0001_1_00_000000 ], longestDurationTasks=[ task_1463520607727_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:07,958 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:07,958 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520607727_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:07,958 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:07,958 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607727_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520607727_0001_00_000001, stoppedTime=1463520607958, exitStatus=0
2016-05-17 21:30:07,959 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520607727_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:07,959 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:07,959 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520607727_0001_1
2016-05-17 21:30:07,959 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607727_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520607727_0001_1
2016-05-17 21:30:07,960 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 48F4F8544E3145C6885FEF9F7F951883 for vertex: vertex_1463520607727_0001_1_00 [A3D32269F5D84EF0B87F047B5D3D4E71]
2016-05-17 21:30:07,967 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520607727_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:07,968 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520607727_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:07,968 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520607727_0001_00_000001, Exiting
2016-05-17 21:30:07,968 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520607727_0001_00_000001
2016-05-17 21:30:07,968 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520607727_0001_00_000001
2016-05-17 21:30:07,969 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520607727_0001_1, queueSize=0
2016-05-17 21:30:07,969 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:07,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520607727_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520607727_0001_1, startTime=1463520607890, finishTime=1463520607967, timeTaken=77, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=398983168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520607946, Process_Duration=5, Process_End_Time=1463520607951, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:07,970 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520607727_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:07,970 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520607727_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:07,970 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520607727_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:30:07 Completed Dag: dag_1463520607727_0001_1
2016-05-17 21:30:07,970 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520607727_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:07,970 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520607727_0001_1_post
2016-05-17 21:30:07,970 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:07,970 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:07,971 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520607727_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:30:07,971 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520607727_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:07,972 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520607727_0001_00_000001 completed successfully
2016-05-17 21:30:07,972 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520607727_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:30:07,977 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...SULTS.0_'field2'_'field2', applicationId=application_1463520607727_0001
2016-05-17 21:30:07,979 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:07,979 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:07,979 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:07,979 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:07,980 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:07,980 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:07,982 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:07,982 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:07,982 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:07,982 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:07,983 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:07,983 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:07,983 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:07,987 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2C61DA7E551A4A348E9E1D2FB3173324/CC6E1976B0DD4024A2BBBDB6ACA9CDFC/.tez/application_1463520607727_0001
2016-05-17 21:30:07,987 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'/_temporary
2016-05-17 21:30:07,988 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.281
2016-05-17 21:30:08,052 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_'field2'
2016-05-17 21:30:08,074 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:08,074 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:08,110 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 21:30:08,121 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-17 21:30:08,121 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:08,121 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:08,122 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:08,143 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:08,144 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS
2016-05-17 21:30:08,157 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:08,157 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:08,157 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS"]
2016-05-17 21:30:08,158 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:08,158 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:08,158 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:08,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-17 21:30:08,159 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:08,171 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:08,176 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:08,177 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:08,177 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:08,186 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/327C86814BE84AF9BAA310549FD0F468/F41273B3BD294BDDA54A329619F79B47/.tez/application_1463520608176_0001 doesn't exist and is created
2016-05-17 21:30:08,200 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:08,201 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:08,201 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/327C86814BE84AF9BAA310549FD0F468/F41273B3BD294BDDA54A329619F79B47/.tez/application_1463520608176_0001
2016-05-17 21:30:08,204 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520608176_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:08,214 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:08,214 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:08,216 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:08,217 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:08,217 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/327C86814BE84AF9BAA310549FD0F468/F41273B3BD294BDDA54A329619F79B47/.tez/application_1463520608176_0001/localmode-log-dir
2016-05-17 21:30:08,218 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/327C86814BE84AF9BAA310549FD0F468/F41273B3BD294BDDA54A329619F79B47/.tez/application_1463520608176_0001/localmode-log-dir/history.txt.appattempt_1463520608176_0001_000000, maxErrors=10
2016-05-17 21:30:08,218 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:08,218 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520608176_0001_000000, appSubmitTime=1463520608203, launchTime=1463520608204
2016-05-17 21:30:08,223 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:08,225 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:08,228 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:08,229 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48185
2016-05-17 21:30:08,230 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:08,231 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:08,233 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520608176_0001_000000, startTime=1463520608233
2016-05-17 21:30:08,234 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:08,233 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:08,301 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:08,301 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:08,301 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ESULTS.0_'field2'_RESULTS, applicationId=application_1463520608176_0001, dagName=[327C86814BE84AF9BAA310549FD0F468/F41273B3BD294BDDA54A329619F79B47] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-17 21:30:08,318 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [327C86814BE84AF9BAA310549FD0F468/F41273B3BD294BDDA54A329619F79B47] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-17 21:30:08,320 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520608176_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/327C86814BE84AF9BAA310549FD0F468/F41273B3BD294BDDA54A329619F79B47/.tez/application_1463520608176_0001/localmode-log-dir/dag_1463520608176_0001_1.dot
2016-05-17 21:30:08,321 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/327C86814BE84AF9BAA310549FD0F468/F41273B3BD294BDDA54A329619F79B47/.tez/application_1463520608176_0001/localmode-log-dir/dag_1463520608176_0001_1-tez-dag.pb.txt
2016-05-17 21:30:08,336 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:08,336 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:08,342 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520608176_0001_1
2016-05-17 21:30:08,342 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:08,342 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [327C86814BE84AF9BAA310549FD0F468/F41273B3BD294BDDA54A329619F79B47] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-17 21:30:08 Running Dag: dag_1463520608176_0001_1
2016-05-17 21:30:08,356 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:08,357 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608176_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520608176_0001_1, submitTime=1463520608318
2016-05-17 21:30:08,357 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FEF97DFF610E41E389A3EE356F562EE9
2016-05-17 21:30:08,357 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:08,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608176_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520608176_0001_1, initTime=1463520608357
2016-05-17 21:30:08,358 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520608176_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:08,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608176_0001_1][Event:DAG_STARTED]: dagID=dag_1463520608176_0001_1, startTime=1463520608359
2016-05-17 21:30:08,359 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:08,360 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520608176_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:08,360 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FEF97DFF610E41E389A3EE356F562EE9 : {CAB7DEB172894A7DB50BDC30A3981B6E={InputName=CAB7DEB172894A7DB50BDC30A3981B6E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:08,360 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ESULTS.0_'field2'_RESULTS, applicationId=application_1463520608176_0001, dagName=[327C86814BE84AF9BAA310549FD0F468/F41273B3BD294BDDA54A329619F79B47] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-17 21:30:08,358 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:08,361 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:08,360 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: CAB7DEB172894A7DB50BDC30A3981B6E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:08,361 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9]
2016-05-17 21:30:08,361 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9]
2016-05-17 21:30:08,361 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9]
2016-05-17 21:30:08,362 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9]. Starting root input initializers: 1
2016-05-17 21:30:08,362 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:08,363 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: CAB7DEB172894A7DB50BDC30A3981B6E on vertex vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9]
2016-05-17 21:30:08,365 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:08,365 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input CAB7DEB172894A7DB50BDC30A3981B6E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:08,366 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:08,367 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:08,368 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:08,368 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: CAB7DEB172894A7DB50BDC30A3981B6E on vertex vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9]
2016-05-17 21:30:08,369 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:08,369 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {CAB7DEB172894A7DB50BDC30A3981B6E=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:08,369 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9] parallelism set to 1
2016-05-17 21:30:08,370 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:08,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608176_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520608176_0001_1_00, eventCount=1
2016-05-17 21:30:08,370 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9]
2016-05-17 21:30:08,371 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=84DFAB8D4A5448F38C65660E1A9434A6, vertexId=vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:08,371 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=84DFAB8D4A5448F38C65660E1A9434A6, vertexId=vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9]
2016-05-17 21:30:08,373 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=84DFAB8D4A5448F38C65660E1A9434A6, vertexName=FEF97DFF610E41E389A3EE356F562EE9, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:08,374 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=84DFAB8D4A5448F38C65660E1A9434A6, vertexId=vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9]
2016-05-17 21:30:08,375 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608176_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FEF97DFF610E41E389A3EE356F562EE9, vertexId=vertex_1463520608176_0001_1_00, initRequestedTime=1463520608360, initedTime=1463520608375, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:08,375 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9] with distanceFromRoot: 0
2016-05-17 21:30:08,375 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520608176_0001, with dag id: dag_1463520608176_0001_1
2016-05-17 21:30:08,375 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:08,375 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:08,376 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FEF97DFF610E41E389A3EE356F562EE9
2016-05-17 21:30:08,376 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9]
2016-05-17 21:30:08,377 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608176_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520608176_0001_1_00, startRequestedTime=1463520608376, startedTime=1463520608376
2016-05-17 21:30:08,377 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:08,377 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:08,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:08,378 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608176_0001_1][Event:TASK_STARTED]: vertexName=FEF97DFF610E41E389A3EE356F562EE9, taskId=task_1463520608176_0001_1_00_000000, scheduledTime=1463520608378, launchTime=1463520608378
2016-05-17 21:30:08,378 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520608176_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:08,379 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9]
2016-05-17 21:30:08,379 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [327C86814BE84AF9BAA310549FD0F468/F41273B3BD294BDDA54A329619F79B47] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS, VertexName: FEF97DFF610E41E389A3EE356F562EE9, VertexParallelism: 1, TaskAttemptID:attempt_1463520608176_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CAB7DEB172894A7DB50BDC30A3981B6E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=84DFAB8D4A5448F38C65660E1A9434A6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:08,379 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520608176_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:08,380 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:08,382 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:08,382 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:08,383 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:08,383 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520608176_0001_00_000001
2016-05-17 21:30:08,383 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520608176_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:08,385 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520608176_0001_00_000001
2016-05-17 21:30:08,385 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:08,385 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520608176_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:08,386 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:08,386 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608176_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520608176_0001_00_000001, launchTime=1463520608386
2016-05-17 21:30:08,386 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520608176_0001_00_000001
2016-05-17 21:30:08,388 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520608176_0001_1_00_000000_0] to container: [container_1463520608176_0001_00_000001]
2016-05-17 21:30:08,388 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520608176_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:08,388 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520608176_0001_1_00_000000_0] started. Is using containerId: [container_1463520608176_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:08,388 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608176_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FEF97DFF610E41E389A3EE356F562EE9, taskAttemptId=attempt_1463520608176_0001_1_00_000000_0, startTime=1463520608388, containerId=container_1463520608176_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520608176_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:08,389 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520608176_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:08,389 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520608176_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:08,389 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:08,390 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520608176_0001_00_000001 given task: attempt_1463520608176_0001_1_00_000000_0
2016-05-17 21:30:08,390 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520608176_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520608176_0001_1_00_000000_0
2016-05-17 21:30:08 Starting to run new task attempt: attempt_1463520608176_0001_1_00_000000_0
2016-05-17 21:30:08,390 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520608176_0001_1_00_000000_0
2016-05-17 21:30:08,390 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:08,390 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:08,390 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:08,390 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:08,390 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:08,391 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:08,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [327C86814BE84AF9BAA310549FD0F468/F41273B3BD294BDDA54A329619F79B47] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS, VertexName: FEF97DFF610E41E389A3EE356F562EE9, VertexParallelism: 1, TaskAttemptID:attempt_1463520608176_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CAB7DEB172894A7DB50BDC30A3981B6E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=84DFAB8D4A5448F38C65660E1A9434A6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:08,391 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:08,399 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520608176_0001_1_00_000000_0
2016-05-17 21:30:08,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:08,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:08,400 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520608176_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:08,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CAB7DEB172894A7DB50BDC30A3981B6E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:08,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:08,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:08,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CAB7DEB172894A7DB50BDC30A3981B6E
2016-05-17 21:30:08,402 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: CAB7DEB172894A7DB50BDC30A3981B6E
2016-05-17 21:30:08,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=84DFAB8D4A5448F38C65660E1A9434A6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:08,403 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:08,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:08,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 84DFAB8D4A5448F38C65660E1A9434A6
2016-05-17 21:30:08,405 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:08,405 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 84DFAB8D4A5448F38C65660E1A9434A6
2016-05-17 21:30:08,408 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:08,409 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: CAB7DEB172894A7DB50BDC30A3981B6E
2016-05-17 21:30:08,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CAB7DEB172894A7DB50BDC30A3981B6E
2016-05-17 21:30:08,412 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FEF97DFF610E41E389A3EE356F562EE9, ordinal: 0
2016-05-17 21:30:08,412 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FEF97DFF610E41E389A3EE356F562EE9, mem on start (mb), free: 230, total: 380, max: 672
2016-05-17 21:30:08,412 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:08,412 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:08,413 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:08,417 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:08,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 84DFAB8D4A5448F38C65660E1A9434A6
2016-05-17 21:30:08,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:08,418 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:08,418 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:08,419 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:08,419 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:08,419 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:08,419 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:08,419 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:08,420 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CAB7DEB172894A7DB50BDC30A3981B6E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:08,420 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 84DFAB8D4A5448F38C65660E1A9434A6, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:08,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:08,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CAB7DEB172894A7DB50BDC30A3981B6E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:08,421 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:08,421 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:08,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CAB7DEB172894A7DB50BDC30A3981B6E
2016-05-17 21:30:08,422 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:08,422 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CAB7DEB172894A7DB50BDC30A3981B6E
2016-05-17 21:30:08,422 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:08,422 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520608176_0001_1_00_000000_0
2016-05-17 21:30:08,423 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:08,426 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:08,427 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: CAB7DEB172894A7DB50BDC30A3981B6E
2016-05-17 21:30:08,427 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS"], id: 84DFAB8D4A5448F38C65660E1A9434A6
2016-05-17 21:30:08,428 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6af30e59
2016-05-17 21:30:08,428 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS"]
2016-05-17 21:30:08,428 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:08,429 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:08,428 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6af30e59
2016-05-17 21:30:08,429 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:08,429 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:08,430 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FEF97DFF610E41E389A3EE356F562EE9, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:30:08,434 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:08,434 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:08,434 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520608176_0001_1_00_000000_0
2016-05-17 21:30:08,434 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520608176_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:08,435 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520608176_0001_1_00_000000_0
2016-05-17 21:30:08,435 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FEF97DFF610E41E389A3EE356F562EE9, mem on close (mb), free: 228, total: 380, max: 672
2016-05-17 21:30:08 Completed running task attempt: attempt_1463520608176_0001_1_00_000000_0
2016-05-17 21:30:08,436 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520608176_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:08,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=398983168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520608430, Process_Duration=5, Process_End_Time=1463520608435, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:08,438 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:08,438 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:08,438 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:08,439 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520608176_0001_00_000001
2016-05-17 21:30:08,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608176_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FEF97DFF610E41E389A3EE356F562EE9, taskAttemptId=attempt_1463520608176_0001_1_00_000000_0, startTime=1463520608388, finishTime=1463520608437, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=398983168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520608430, Process_Duration=5, Process_End_Time=1463520608435, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:08,439 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520608176_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:08,439 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:08,439 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520608176_0001_1_00_000000_0
2016-05-17 21:30:08,440 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:08,440 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608176_0001_1][Event:TASK_FINISHED]: vertexName=FEF97DFF610E41E389A3EE356F562EE9, taskId=task_1463520608176_0001_1_00_000000, startTime=1463520608388, finishTime=1463520608439, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1463520608176_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=398983168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520608430, Process_Duration=5, Process_End_Time=1463520608435, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:08,440 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520608176_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:08,440 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9] : 1
2016-05-17 21:30:08,441 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9]
2016-05-17 21:30:08,442 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:08,442 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608176_0001_1][Event:VERTEX_FINISHED]: vertexName=FEF97DFF610E41E389A3EE356F562EE9, vertexId=vertex_1463520608176_0001_1_00, initRequestedTime=1463520608360, initedTime=1463520608375, startRequestedTime=1463520608376, startedTime=1463520608376, finishTime=1463520608441, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=398983168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520608430, Process_Duration=5, Process_End_Time=1463520608435, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520608388, firstTasksToStart=[ task_1463520608176_0001_1_00_000000 ], lastTaskFinishTime=1463520608437, lastTasksToFinish=[ task_1463520608176_0001_1_00_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520608176_0001_1_00_000000 ], longestDurationTasks=[ task_1463520608176_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:08,442 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:08,442 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520608176_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:08,442 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608176_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520608176_0001_00_000001, stoppedTime=1463520608442, exitStatus=0
2016-05-17 21:30:08,443 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520608176_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:08,443 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:08,443 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520608176_0001_1
2016-05-17 21:30:08,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608176_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520608176_0001_1
2016-05-17 21:30:08,444 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 84DFAB8D4A5448F38C65660E1A9434A6 for vertex: vertex_1463520608176_0001_1_00 [FEF97DFF610E41E389A3EE356F562EE9]
2016-05-17 21:30:08,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:08,449 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520608176_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:08,449 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520608176_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:08,449 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520608176_0001_00_000001, Exiting
2016-05-17 21:30:08,450 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520608176_0001_00_000001
2016-05-17 21:30:08,450 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520608176_0001_00_000001
2016-05-17 21:30:08,451 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520608176_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:08,451 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520608176_0001_00_000001 completed successfully
2016-05-17 21:30:08,455 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520608176_0001_1, queueSize=0
2016-05-17 21:30:08,455 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:08,456 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608176_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520608176_0001_1, startTime=1463520608359, finishTime=1463520608454, timeTaken=95, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=398983168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520608430, Process_Duration=5, Process_End_Time=1463520608435, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:08,456 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520608176_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:08,456 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520608176_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:08,456 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520608176_0001_00_000001
2016-05-17 21:30:08,457 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520608176_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:08 Completed Dag: dag_1463520608176_0001_1
2016-05-17 21:30:08,458 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520608176_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:08,458 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520608176_0001_1_post
2016-05-17 21:30:08,459 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:08,459 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:08,462 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ESULTS.0_'field2'_RESULTS, applicationId=application_1463520608176_0001
2016-05-17 21:30:08,464 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:08,464 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:08,464 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:08,464 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:08,464 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:08,465 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:08,465 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:08,465 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:08,465 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:08,466 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:08,466 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:08,466 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:08,466 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:08,469 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/327C86814BE84AF9BAA310549FD0F468/F41273B3BD294BDDA54A329619F79B47/.tez/application_1463520608176_0001
2016-05-17 21:30:08,470 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS/_temporary
2016-05-17 21:30:08,470 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.313
2016-05-17 21:30:08,538 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_RESULTS
2016-05-17 21:30:08,561 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:08,561 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:08,596 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 21:30:08,601 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-17 21:30:08,601 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:08,601 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:08,602 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:08,615 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:08,615 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL
2016-05-17 21:30:08,624 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:08,625 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:08,625 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL"]
2016-05-17 21:30:08,625 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:08,625 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:08,625 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:08,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-17 21:30:08,626 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:08,638 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:08,642 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:08,642 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:08,642 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:08,651 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6A1F4598A4B54B698CAC1DECB4773B39/F463B63F688B437F8A0E91A8CA06A7D7/.tez/application_1463520608642_0001 doesn't exist and is created
2016-05-17 21:30:08,666 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:08,666 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:08,666 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6A1F4598A4B54B698CAC1DECB4773B39/F463B63F688B437F8A0E91A8CA06A7D7/.tez/application_1463520608642_0001
2016-05-17 21:30:08,669 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520608642_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:08,678 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:08,679 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:08,680 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:08,680 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:08,681 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6A1F4598A4B54B698CAC1DECB4773B39/F463B63F688B437F8A0E91A8CA06A7D7/.tez/application_1463520608642_0001/localmode-log-dir
2016-05-17 21:30:08,681 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6A1F4598A4B54B698CAC1DECB4773B39/F463B63F688B437F8A0E91A8CA06A7D7/.tez/application_1463520608642_0001/localmode-log-dir/history.txt.appattempt_1463520608642_0001_000000, maxErrors=10
2016-05-17 21:30:08,681 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:08,681 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520608642_0001_000000, appSubmitTime=1463520608668, launchTime=1463520608669
2016-05-17 21:30:08,685 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:08,686 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:08,686 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:08,687 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:08,690 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41330
2016-05-17 21:30:08,691 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:08,693 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520608642_0001_000000, startTime=1463520608693
2016-05-17 21:30:08,693 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:08,693 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:08,766 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:08,766 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:08,766 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_'field2'_ALL, applicationId=application_1463520608642_0001, dagName=[6A1F4598A4B54B698CAC1DECB4773B39/F463B63F688B437F8A0E91A8CA06A7D7] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-17 21:30:08,783 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [6A1F4598A4B54B698CAC1DECB4773B39/F463B63F688B437F8A0E91A8CA06A7D7] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-17 21:30:08,785 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520608642_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6A1F4598A4B54B698CAC1DECB4773B39/F463B63F688B437F8A0E91A8CA06A7D7/.tez/application_1463520608642_0001/localmode-log-dir/dag_1463520608642_0001_1.dot
2016-05-17 21:30:08,785 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6A1F4598A4B54B698CAC1DECB4773B39/F463B63F688B437F8A0E91A8CA06A7D7/.tez/application_1463520608642_0001/localmode-log-dir/dag_1463520608642_0001_1-tez-dag.pb.txt
2016-05-17 21:30:08,793 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520608642_0001_1
2016-05-17 21:30:08,793 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:08,794 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [6A1F4598A4B54B698CAC1DECB4773B39/F463B63F688B437F8A0E91A8CA06A7D7] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-17 21:30:08 Running Dag: dag_1463520608642_0001_1
2016-05-17 21:30:08,806 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608642_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520608642_0001_1, submitTime=1463520608783
2016-05-17 21:30:08,806 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:08,807 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E4F33893E7CB414891FE79D109663CA3
2016-05-17 21:30:08,807 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:08,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608642_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520608642_0001_1, initTime=1463520608806
2016-05-17 21:30:08,808 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:08,808 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520608642_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:08,810 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_'field2'_ALL, applicationId=application_1463520608642_0001, dagName=[6A1F4598A4B54B698CAC1DECB4773B39/F463B63F688B437F8A0E91A8CA06A7D7] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-17 21:30:08,810 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608642_0001_1][Event:DAG_STARTED]: dagID=dag_1463520608642_0001_1, startTime=1463520608809
2016-05-17 21:30:08,810 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:08,811 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520608642_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:08,811 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E4F33893E7CB414891FE79D109663CA3 : {7F344158F2194065977F111961399F7A={InputName=7F344158F2194065977F111961399F7A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:08,811 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 7F344158F2194065977F111961399F7A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:08,811 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3]
2016-05-17 21:30:08,811 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3]
2016-05-17 21:30:08,811 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3]
2016-05-17 21:30:08,812 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3]. Starting root input initializers: 1
2016-05-17 21:30:08,810 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:08,812 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:08,812 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 7F344158F2194065977F111961399F7A on vertex vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3]
2016-05-17 21:30:08,814 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:08,814 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 7F344158F2194065977F111961399F7A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:08,815 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:08,815 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:08,816 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:08,816 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 7F344158F2194065977F111961399F7A on vertex vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3]
2016-05-17 21:30:08,817 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:08,817 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {7F344158F2194065977F111961399F7A=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:08,817 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3] parallelism set to 1
2016-05-17 21:30:08,817 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:08,817 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608642_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520608642_0001_1_00, eventCount=1
2016-05-17 21:30:08,818 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3]
2016-05-17 21:30:08,818 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D34D4DDC69C94D248D7989B32962E76A, vertexId=vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:08,818 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D34D4DDC69C94D248D7989B32962E76A, vertexId=vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3]
2016-05-17 21:30:08,819 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D34D4DDC69C94D248D7989B32962E76A, vertexName=E4F33893E7CB414891FE79D109663CA3, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:08,820 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D34D4DDC69C94D248D7989B32962E76A, vertexId=vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3]
2016-05-17 21:30:08,820 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:08,822 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608642_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E4F33893E7CB414891FE79D109663CA3, vertexId=vertex_1463520608642_0001_1_00, initRequestedTime=1463520608811, initedTime=1463520608820, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:08,822 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3] with distanceFromRoot: 0
2016-05-17 21:30:08,822 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:08,822 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E4F33893E7CB414891FE79D109663CA3
2016-05-17 21:30:08,823 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3]
2016-05-17 21:30:08,823 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:08,823 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608642_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520608642_0001_1_00, startRequestedTime=1463520608822, startedTime=1463520608822
2016-05-17 21:30:08,824 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:08,824 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:08,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608642_0001_1][Event:TASK_STARTED]: vertexName=E4F33893E7CB414891FE79D109663CA3, taskId=task_1463520608642_0001_1_00_000000, scheduledTime=1463520608824, launchTime=1463520608824
2016-05-17 21:30:08,825 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520608642_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:08,825 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3]
2016-05-17 21:30:08,825 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6A1F4598A4B54B698CAC1DECB4773B39/F463B63F688B437F8A0E91A8CA06A7D7] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL, VertexName: E4F33893E7CB414891FE79D109663CA3, VertexParallelism: 1, TaskAttemptID:attempt_1463520608642_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7F344158F2194065977F111961399F7A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D34D4DDC69C94D248D7989B32962E76A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:08,825 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520608642_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:08,826 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:08,826 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:08,827 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:08,827 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:08,827 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520608642_0001_00_000001
2016-05-17 21:30:08,828 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520608642_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:08,828 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520608642_0001, with dag id: dag_1463520608642_0001_1
2016-05-17 21:30:08,829 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520608642_0001_00_000001
2016-05-17 21:30:08,829 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:08,830 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:08,830 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520608642_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:08,831 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608642_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520608642_0001_00_000001, launchTime=1463520608830
2016-05-17 21:30:08,831 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520608642_0001_00_000001
2016-05-17 21:30:08,833 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520608642_0001_1_00_000000_0] to container: [container_1463520608642_0001_00_000001]
2016-05-17 21:30:08,833 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520608642_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:08,834 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520608642_0001_1_00_000000_0] started. Is using containerId: [container_1463520608642_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:08,834 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608642_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E4F33893E7CB414891FE79D109663CA3, taskAttemptId=attempt_1463520608642_0001_1_00_000000_0, startTime=1463520608834, containerId=container_1463520608642_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520608642_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:08,834 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:08,834 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520608642_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:08,835 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520608642_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:08,835 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520608642_0001_00_000001 given task: attempt_1463520608642_0001_1_00_000000_0
2016-05-17 21:30:08,835 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520608642_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520608642_0001_1_00_000000_0
2016-05-17 21:30:08 Starting to run new task attempt: attempt_1463520608642_0001_1_00_000000_0
2016-05-17 21:30:08,836 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520608642_0001_1_00_000000_0
2016-05-17 21:30:08,836 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:08,836 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:08,836 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:08,837 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:08,837 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:08,837 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:08,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6A1F4598A4B54B698CAC1DECB4773B39/F463B63F688B437F8A0E91A8CA06A7D7] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL, VertexName: E4F33893E7CB414891FE79D109663CA3, VertexParallelism: 1, TaskAttemptID:attempt_1463520608642_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7F344158F2194065977F111961399F7A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D34D4DDC69C94D248D7989B32962E76A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:08,838 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:08,838 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520608642_0001_1_00_000000_0
2016-05-17 21:30:08,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:08,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:08,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:08,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=7F344158F2194065977F111961399F7A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:08,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:08,841 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 7F344158F2194065977F111961399F7A
2016-05-17 21:30:08,841 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 7F344158F2194065977F111961399F7A
2016-05-17 21:30:08,844 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D34D4DDC69C94D248D7989B32962E76A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:08,845 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520608642_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:08,845 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:08,846 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 7F344158F2194065977F111961399F7A
2016-05-17 21:30:08,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 7F344158F2194065977F111961399F7A
2016-05-17 21:30:08,845 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:08,846 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:08,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D34D4DDC69C94D248D7989B32962E76A
2016-05-17 21:30:08,848 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:08,848 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D34D4DDC69C94D248D7989B32962E76A
2016-05-17 21:30:08,853 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E4F33893E7CB414891FE79D109663CA3, ordinal: 0
2016-05-17 21:30:08,853 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4F33893E7CB414891FE79D109663CA3, mem on start (mb), free: 155, total: 380, max: 672
2016-05-17 21:30:08,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:08,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:08,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:08,857 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:08,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D34D4DDC69C94D248D7989B32962E76A
2016-05-17 21:30:08,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:08,858 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:08,858 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:08,858 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:08,858 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:08,858 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:08,858 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:08,858 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:08,858 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 7F344158F2194065977F111961399F7A, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:08,859 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D34D4DDC69C94D248D7989B32962E76A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:08,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:08,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 7F344158F2194065977F111961399F7A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:08,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 7F344158F2194065977F111961399F7A
2016-05-17 21:30:08,859 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:08,860 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 7F344158F2194065977F111961399F7A
2016-05-17 21:30:08,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:08,860 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:08,860 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:08,861 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520608642_0001_1_00_000000_0
2016-05-17 21:30:08,861 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:08,861 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@34e85aab
2016-05-17 21:30:08,862 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@34e85aab
2016-05-17 21:30:08,862 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:08,862 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:08,864 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:08,864 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 7F344158F2194065977F111961399F7A
2016-05-17 21:30:08,865 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL"], id: D34D4DDC69C94D248D7989B32962E76A
2016-05-17 21:30:08,865 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL"]
2016-05-17 21:30:08,865 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:08,865 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:08,866 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E4F33893E7CB414891FE79D109663CA3, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:08,868 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:08,869 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:08,869 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520608642_0001_1_00_000000_0
2016-05-17 21:30:08,869 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520608642_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:08,870 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520608642_0001_1_00_000000_0
2016-05-17 21:30:08,870 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4F33893E7CB414891FE79D109663CA3, mem on close (mb), free: 154, total: 380, max: 672
2016-05-17 21:30:08 Completed running task attempt: attempt_1463520608642_0001_1_00_000000_0
2016-05-17 21:30:08,871 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520608642_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:08,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=398983168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520608866, Process_Duration=4, Process_End_Time=1463520608870, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:08,872 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608642_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E4F33893E7CB414891FE79D109663CA3, taskAttemptId=attempt_1463520608642_0001_1_00_000000_0, startTime=1463520608834, finishTime=1463520608872, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=398983168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520608866, Process_Duration=4, Process_End_Time=1463520608870, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:08,872 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:08,873 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520608642_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:08,872 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:08,873 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:08,873 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520608642_0001_1_00_000000_0
2016-05-17 21:30:08,873 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:08,875 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520608642_0001_00_000001
2016-05-17 21:30:08,875 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:08,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608642_0001_1][Event:TASK_FINISHED]: vertexName=E4F33893E7CB414891FE79D109663CA3, taskId=task_1463520608642_0001_1_00_000000, startTime=1463520608834, finishTime=1463520608873, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1463520608642_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=398983168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520608866, Process_Duration=4, Process_End_Time=1463520608870, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:08,875 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520608642_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:08,876 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3] : 1
2016-05-17 21:30:08,876 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3]
2016-05-17 21:30:08,877 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:08,877 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608642_0001_1][Event:VERTEX_FINISHED]: vertexName=E4F33893E7CB414891FE79D109663CA3, vertexId=vertex_1463520608642_0001_1_00, initRequestedTime=1463520608811, initedTime=1463520608820, startRequestedTime=1463520608822, startedTime=1463520608822, finishTime=1463520608876, timeTaken=54, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=398983168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520608866, Process_Duration=4, Process_End_Time=1463520608870, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520608834, firstTasksToStart=[ task_1463520608642_0001_1_00_000000 ], lastTaskFinishTime=1463520608872, lastTasksToFinish=[ task_1463520608642_0001_1_00_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520608642_0001_1_00_000000 ], longestDurationTasks=[ task_1463520608642_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:08,877 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:08,878 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520608642_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:08,878 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608642_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520608642_0001_00_000001, stoppedTime=1463520608878, exitStatus=0
2016-05-17 21:30:08,878 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:08,878 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520608642_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:08,878 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:08,878 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520608642_0001_1
2016-05-17 21:30:08,879 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608642_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520608642_0001_1
2016-05-17 21:30:08,879 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D34D4DDC69C94D248D7989B32962E76A for vertex: vertex_1463520608642_0001_1_00 [E4F33893E7CB414891FE79D109663CA3]
2016-05-17 21:30:08,886 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520608642_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:08,886 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520608642_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:08,886 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520608642_0001_00_000001, Exiting
2016-05-17 21:30:08,886 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520608642_0001_00_000001
2016-05-17 21:30:08,886 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520608642_0001_00_000001
2016-05-17 21:30:08,887 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520608642_0001_1, queueSize=0
2016-05-17 21:30:08,887 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520608642_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:08,887 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:08,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520608642_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520608642_0001_1, startTime=1463520608809, finishTime=1463520608885, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=398983168, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520608866, Process_Duration=4, Process_End_Time=1463520608870, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:08,888 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520608642_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:08,888 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520608642_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:08,889 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520608642_0001_00_000001
2016-05-17 21:30:08,888 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520608642_0001_00_000001 completed successfully
2016-05-17 21:30:08 Completed Dag: dag_1463520608642_0001_1
2016-05-17 21:30:08,892 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520608642_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:08,893 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520608642_0001_1_post
2016-05-17 21:30:08,893 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:08,893 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:08,893 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520608642_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:30:08,894 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520608642_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:30:08,898 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.0_'field2'_ALL, applicationId=application_1463520608642_0001
2016-05-17 21:30:08,899 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:08,899 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:08,899 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:08,900 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:08,900 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:08,900 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:08,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:08,901 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:08,901 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:08,901 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:08,901 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:08,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:08,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:08,906 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6A1F4598A4B54B698CAC1DECB4773B39/F463B63F688B437F8A0E91A8CA06A7D7/.tez/application_1463520608642_0001
2016-05-17 21:30:08,906 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL/_temporary
2016-05-17 21:30:08,907 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.282
2016-05-17 21:30:08,969 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_ALL
2016-05-17 21:30:08,990 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:08,990 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:09,044 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 21:30:09,049 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-17 21:30:09,050 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:09,050 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:09,050 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:09,052 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:09,052 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:09,065 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:09,065 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1
2016-05-17 21:30:09,074 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:09,074 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:09,074 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1"]
2016-05-17 21:30:09,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:09,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:09,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:09,076 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-17 21:30:09,076 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:09,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:09,091 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:09,092 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:09,093 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:09,103 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/23E178BCBD99461691F502C53745D5F9/D4D13B0E6330400BAD0F09A7C2A3E3C9/.tez/application_1463520609092_0001 doesn't exist and is created
2016-05-17 21:30:09,120 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:09,120 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:09,121 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/23E178BCBD99461691F502C53745D5F9/D4D13B0E6330400BAD0F09A7C2A3E3C9/.tez/application_1463520609092_0001
2016-05-17 21:30:09,123 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520609092_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:09,133 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:09,133 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:09,134 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:09,135 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:09,135 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/23E178BCBD99461691F502C53745D5F9/D4D13B0E6330400BAD0F09A7C2A3E3C9/.tez/application_1463520609092_0001/localmode-log-dir
2016-05-17 21:30:09,135 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/23E178BCBD99461691F502C53745D5F9/D4D13B0E6330400BAD0F09A7C2A3E3C9/.tez/application_1463520609092_0001/localmode-log-dir/history.txt.appattempt_1463520609092_0001_000000, maxErrors=10
2016-05-17 21:30:09,135 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:09,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520609092_0001_000000, appSubmitTime=1463520609123, launchTime=1463520609123
2016-05-17 21:30:09,139 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:09,141 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:09,142 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:09,142 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:09,146 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:37801
2016-05-17 21:30:09,149 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:09,151 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520609092_0001_000000, startTime=1463520609151
2016-05-17 21:30:09,151 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:09,151 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:09,221 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:09,221 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:09,221 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, applicationId=application_1463520609092_0001, dagName=[23E178BCBD99461691F502C53745D5F9/D4D13B0E6330400BAD0F09A7C2A3E3C9] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-17 21:30:09,237 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [23E178BCBD99461691F502C53745D5F9/D4D13B0E6330400BAD0F09A7C2A3E3C9] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-17 21:30:09,240 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520609092_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/23E178BCBD99461691F502C53745D5F9/D4D13B0E6330400BAD0F09A7C2A3E3C9/.tez/application_1463520609092_0001/localmode-log-dir/dag_1463520609092_0001_1.dot
2016-05-17 21:30:09,240 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/23E178BCBD99461691F502C53745D5F9/D4D13B0E6330400BAD0F09A7C2A3E3C9/.tez/application_1463520609092_0001/localmode-log-dir/dag_1463520609092_0001_1-tez-dag.pb.txt
2016-05-17 21:30:09,249 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520609092_0001_1
2016-05-17 21:30:09,249 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:09,249 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [23E178BCBD99461691F502C53745D5F9/D4D13B0E6330400BAD0F09A7C2A3E3C9] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-17 21:30:09 Running Dag: dag_1463520609092_0001_1
2016-05-17 21:30:09,263 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609092_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520609092_0001_1, submitTime=1463520609238
2016-05-17 21:30:09,263 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:09,264 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DA548ADDD6F341FDA7EC519168AABBF5
2016-05-17 21:30:09,264 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:09,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:09,265 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609092_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520609092_0001_1, initTime=1463520609264
2016-05-17 21:30:09,266 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520609092_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:09,267 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609092_0001_1][Event:DAG_STARTED]: dagID=dag_1463520609092_0001_1, startTime=1463520609266
2016-05-17 21:30:09,267 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:09,268 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, applicationId=application_1463520609092_0001, dagName=[23E178BCBD99461691F502C53745D5F9/D4D13B0E6330400BAD0F09A7C2A3E3C9] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-17 21:30:09,267 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:09,268 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520609092_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:09,269 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DA548ADDD6F341FDA7EC519168AABBF5 : {E6F8259A0906470AB80C4DADF9C6F864={InputName=E6F8259A0906470AB80C4DADF9C6F864}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:09,269 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E6F8259A0906470AB80C4DADF9C6F864, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:09,269 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5]
2016-05-17 21:30:09,269 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5]
2016-05-17 21:30:09,269 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5]
2016-05-17 21:30:09,269 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5]. Starting root input initializers: 1
2016-05-17 21:30:09,275 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:09,276 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E6F8259A0906470AB80C4DADF9C6F864 on vertex vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5]
2016-05-17 21:30:09,279 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:09,279 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E6F8259A0906470AB80C4DADF9C6F864 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:09,281 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:09,281 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:09,284 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:09,284 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E6F8259A0906470AB80C4DADF9C6F864 on vertex vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5]
2016-05-17 21:30:09,285 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:09,286 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E6F8259A0906470AB80C4DADF9C6F864=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:09,286 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5] parallelism set to 1
2016-05-17 21:30:09,286 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:09,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609092_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520609092_0001_1_00, eventCount=1
2016-05-17 21:30:09,286 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5]
2016-05-17 21:30:09,287 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=5B690FD9C9E94A879DD74646D787C629, vertexId=vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:09,287 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=5B690FD9C9E94A879DD74646D787C629, vertexId=vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5]
2016-05-17 21:30:09,289 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=5B690FD9C9E94A879DD74646D787C629, vertexName=DA548ADDD6F341FDA7EC519168AABBF5, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:09,289 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=5B690FD9C9E94A879DD74646D787C629, vertexId=vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5]
2016-05-17 21:30:09,291 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609092_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DA548ADDD6F341FDA7EC519168AABBF5, vertexId=vertex_1463520609092_0001_1_00, initRequestedTime=1463520609269, initedTime=1463520609291, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:09,293 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5] with distanceFromRoot: 0
2016-05-17 21:30:09,293 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:09,293 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DA548ADDD6F341FDA7EC519168AABBF5
2016-05-17 21:30:09,294 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5]
2016-05-17 21:30:09,294 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609092_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520609092_0001_1_00, startRequestedTime=1463520609293, startedTime=1463520609293
2016-05-17 21:30:09,295 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:09,295 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609092_0001_1][Event:TASK_STARTED]: vertexName=DA548ADDD6F341FDA7EC519168AABBF5, taskId=task_1463520609092_0001_1_00_000000, scheduledTime=1463520609295, launchTime=1463520609295
2016-05-17 21:30:09,296 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520609092_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:09,296 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5]
2016-05-17 21:30:09,296 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [23E178BCBD99461691F502C53745D5F9/D4D13B0E6330400BAD0F09A7C2A3E3C9] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, VertexName: DA548ADDD6F341FDA7EC519168AABBF5, VertexParallelism: 1, TaskAttemptID:attempt_1463520609092_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E6F8259A0906470AB80C4DADF9C6F864, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5B690FD9C9E94A879DD74646D787C629, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:09,297 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520609092_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:09,297 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:09,292 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520609092_0001, with dag id: dag_1463520609092_0001_1
2016-05-17 21:30:09,292 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:09,298 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:09,298 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:09,305 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:09,306 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:09,306 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:09,307 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520609092_0001_00_000001
2016-05-17 21:30:09,307 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520609092_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:09,309 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520609092_0001_00_000001
2016-05-17 21:30:09,309 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:09,312 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520609092_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:09,312 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:09,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609092_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520609092_0001_00_000001, launchTime=1463520609312
2016-05-17 21:30:09,313 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520609092_0001_00_000001
2016-05-17 21:30:09,314 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520609092_0001_1_00_000000_0] to container: [container_1463520609092_0001_00_000001]
2016-05-17 21:30:09,314 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520609092_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:09,315 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520609092_0001_00_000001 given task: attempt_1463520609092_0001_1_00_000000_0
2016-05-17 21:30:09,315 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520609092_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520609092_0001_1_00_000000_0
2016-05-17 21:30:09,315 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520609092_0001_1_00_000000_0] started. Is using containerId: [container_1463520609092_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:09,316 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:09 Starting to run new task attempt: attempt_1463520609092_0001_1_00_000000_0
2016-05-17 21:30:09,317 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520609092_0001_1_00_000000_0
2016-05-17 21:30:09,317 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:09,317 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:09,317 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:09,317 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:09,318 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:09,318 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:09,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [23E178BCBD99461691F502C53745D5F9/D4D13B0E6330400BAD0F09A7C2A3E3C9] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, VertexName: DA548ADDD6F341FDA7EC519168AABBF5, VertexParallelism: 1, TaskAttemptID:attempt_1463520609092_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E6F8259A0906470AB80C4DADF9C6F864, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5B690FD9C9E94A879DD74646D787C629, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:09,318 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:09,316 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609092_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DA548ADDD6F341FDA7EC519168AABBF5, taskAttemptId=attempt_1463520609092_0001_1_00_000000_0, startTime=1463520609315, containerId=container_1463520609092_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520609092_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:09,320 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520609092_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:09,320 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520609092_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:09,321 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520609092_0001_1_00_000000_0
2016-05-17 21:30:09,321 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:09,321 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:09,324 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E6F8259A0906470AB80C4DADF9C6F864, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:09,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:09,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E6F8259A0906470AB80C4DADF9C6F864
2016-05-17 21:30:09,325 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E6F8259A0906470AB80C4DADF9C6F864
2016-05-17 21:30:09,327 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:09,330 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:09,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=5B690FD9C9E94A879DD74646D787C629, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:09,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:09,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 5B690FD9C9E94A879DD74646D787C629
2016-05-17 21:30:09,338 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:09,338 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 5B690FD9C9E94A879DD74646D787C629
2016-05-17 21:30:09,342 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:09,342 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520609092_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:09,344 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E6F8259A0906470AB80C4DADF9C6F864
2016-05-17 21:30:09,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E6F8259A0906470AB80C4DADF9C6F864
2016-05-17 21:30:09,352 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:09,353 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 5B690FD9C9E94A879DD74646D787C629
2016-05-17 21:30:09,353 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DA548ADDD6F341FDA7EC519168AABBF5, ordinal: 0
2016-05-17 21:30:09,353 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA548ADDD6F341FDA7EC519168AABBF5, mem on start (mb), free: 274, total: 381, max: 672
2016-05-17 21:30:09,353 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:09,353 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:09,353 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:09,353 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:09,353 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:09,354 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:09,354 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:09,354 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:09,354 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:09,355 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:09,355 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:09,355 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E6F8259A0906470AB80C4DADF9C6F864, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:09,355 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 5B690FD9C9E94A879DD74646D787C629, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:09,355 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:09,355 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E6F8259A0906470AB80C4DADF9C6F864
2016-05-17 21:30:09,356 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:09,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E6F8259A0906470AB80C4DADF9C6F864
2016-05-17 21:30:09,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E6F8259A0906470AB80C4DADF9C6F864 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:09,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:09,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:09,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:09,357 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520609092_0001_1_00_000000_0
2016-05-17 21:30:09,360 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:09,360 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E6F8259A0906470AB80C4DADF9C6F864
2016-05-17 21:30:09,361 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1"], id: 5B690FD9C9E94A879DD74646D787C629
2016-05-17 21:30:09,361 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1"]
2016-05-17 21:30:09,361 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:09,361 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:09,361 INFO input.MRInput (MRInput.java:checkAndAwaitRecordReaderInitialization(597)) - Awaiting RecordReader initialization
2016-05-17 21:30:09,362 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:09,362 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7517eb10
2016-05-17 21:30:09,363 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7517eb10
2016-05-17 21:30:09,363 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:09,364 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:09,364 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DA548ADDD6F341FDA7EC519168AABBF5, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:09,367 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:09,368 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:09,368 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520609092_0001_1_00_000000_0
2016-05-17 21:30:09,369 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520609092_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:09,370 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520609092_0001_1_00_000000_0
2016-05-17 21:30:09,370 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA548ADDD6F341FDA7EC519168AABBF5, mem on close (mb), free: 273, total: 381, max: 672
2016-05-17 21:30:09 Completed running task attempt: attempt_1463520609092_0001_1_00_000000_0
2016-05-17 21:30:09,372 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520609092_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:09,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520609364, Process_Duration=6, Process_End_Time=1463520609370, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:09,373 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:09,373 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609092_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DA548ADDD6F341FDA7EC519168AABBF5, taskAttemptId=attempt_1463520609092_0001_1_00_000000_0, startTime=1463520609315, finishTime=1463520609373, timeTaken=58, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520609364, Process_Duration=6, Process_End_Time=1463520609370, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:09,374 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520609092_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:09,373 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:09,374 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520609092_0001_1_00_000000_0
2016-05-17 21:30:09,374 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:09,374 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520609092_0001_00_000001
2016-05-17 21:30:09,375 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609092_0001_1][Event:TASK_FINISHED]: vertexName=DA548ADDD6F341FDA7EC519168AABBF5, taskId=task_1463520609092_0001_1_00_000000, startTime=1463520609315, finishTime=1463520609374, timeTaken=59, status=SUCCEEDED, successfulAttemptID=attempt_1463520609092_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520609364, Process_Duration=6, Process_End_Time=1463520609370, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:09,375 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520609092_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:09,375 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5] : 1
2016-05-17 21:30:09,375 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5]
2016-05-17 21:30:09,375 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:09,376 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:09,376 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:09,378 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609092_0001_1][Event:VERTEX_FINISHED]: vertexName=DA548ADDD6F341FDA7EC519168AABBF5, vertexId=vertex_1463520609092_0001_1_00, initRequestedTime=1463520609269, initedTime=1463520609291, startRequestedTime=1463520609293, startedTime=1463520609293, finishTime=1463520609375, timeTaken=82, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520609364, Process_Duration=6, Process_End_Time=1463520609370, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520609315, firstTasksToStart=[ task_1463520609092_0001_1_00_000000 ], lastTaskFinishTime=1463520609373, lastTasksToFinish=[ task_1463520609092_0001_1_00_000000 ], minTaskDuration=58, maxTaskDuration=58, avgTaskDuration=58.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520609092_0001_1_00_000000 ], longestDurationTasks=[ task_1463520609092_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:09,378 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:09,378 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520609092_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:09,378 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609092_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520609092_0001_00_000001, stoppedTime=1463520609378, exitStatus=0
2016-05-17 21:30:09,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:09,378 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520609092_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:09,379 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:09,379 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520609092_0001_1
2016-05-17 21:30:09,379 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609092_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520609092_0001_1
2016-05-17 21:30:09,379 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 5B690FD9C9E94A879DD74646D787C629 for vertex: vertex_1463520609092_0001_1_00 [DA548ADDD6F341FDA7EC519168AABBF5]
2016-05-17 21:30:09,385 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520609092_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:09,385 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520609092_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:09,386 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520609092_0001_00_000001, Exiting
2016-05-17 21:30:09,386 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520609092_0001_00_000001
2016-05-17 21:30:09,386 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520609092_0001_00_000001
2016-05-17 21:30:09,387 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520609092_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:09,388 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520609092_0001_00_000001 completed successfully
2016-05-17 21:30:09,392 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520609092_0001_1, queueSize=0
2016-05-17 21:30:09,392 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:09,392 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609092_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520609092_0001_1, startTime=1463520609266, finishTime=1463520609390, timeTaken=124, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520609364, Process_Duration=6, Process_End_Time=1463520609370, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:09,392 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520609092_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:09,393 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520609092_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:09,393 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520609092_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:09 Completed Dag: dag_1463520609092_0001_1
2016-05-17 21:30:09,393 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520609092_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:09,393 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520609092_0001_1_post
2016-05-17 21:30:09,393 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:09,393 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:09,394 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520609092_0001_00_000001
2016-05-17 21:30:09,401 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, applicationId=application_1463520609092_0001
2016-05-17 21:30:09,403 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:09,403 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:09,403 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:09,403 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:09,404 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:09,404 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:09,405 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:09,406 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:09,406 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:09,406 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:09,407 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:09,407 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:09,407 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:09,412 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/23E178BCBD99461691F502C53745D5F9/D4D13B0E6330400BAD0F09A7C2A3E3C9/.tez/application_1463520609092_0001
2016-05-17 21:30:09,412 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1/_temporary
2016-05-17 21:30:09,413 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.338
2016-05-17 21:30:09,477 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_-1
2016-05-17 21:30:09,500 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:09,501 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:09,531 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-17 21:30:09,534 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-17 21:30:09,534 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:09,534 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:09,535 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_'field2'
2016-05-17 21:30:09,535 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:09,535 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:09,554 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:09,554 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:09,584 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 21:30:09,589 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 21:30:09,590 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:09,590 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:09,590 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:09,602 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:09,603 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS
2016-05-17 21:30:09,611 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:09,612 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:09,612 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS"]
2016-05-17 21:30:09,612 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:09,612 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:09,612 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:09,614 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-17 21:30:09,614 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:09,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:09,630 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:09,631 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:09,631 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:09,639 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8086ACE92F504F9BBF5D377FF2C5034C/BAF7EEC3B07E4206A3ABB2224017FC20/.tez/application_1463520609631_0001 doesn't exist and is created
2016-05-17 21:30:09,654 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:09,655 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:09,655 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8086ACE92F504F9BBF5D377FF2C5034C/BAF7EEC3B07E4206A3ABB2224017FC20/.tez/application_1463520609631_0001
2016-05-17 21:30:09,657 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520609631_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:09,667 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:09,668 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:09,669 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:09,669 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:09,669 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8086ACE92F504F9BBF5D377FF2C5034C/BAF7EEC3B07E4206A3ABB2224017FC20/.tez/application_1463520609631_0001/localmode-log-dir
2016-05-17 21:30:09,669 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8086ACE92F504F9BBF5D377FF2C5034C/BAF7EEC3B07E4206A3ABB2224017FC20/.tez/application_1463520609631_0001/localmode-log-dir/history.txt.appattempt_1463520609631_0001_000000, maxErrors=10
2016-05-17 21:30:09,670 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:09,670 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520609631_0001_000000, appSubmitTime=1463520609657, launchTime=1463520609657
2016-05-17 21:30:09,675 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43159
2016-05-17 21:30:09,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:09,680 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:09,680 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:09,681 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:09,681 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:09,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520609631_0001_000000, startTime=1463520609683
2016-05-17 21:30:09,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:09,683 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:09,755 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:09,755 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:09,755 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...RESULTS.0_UNKNOWN_RESULTS, applicationId=application_1463520609631_0001, dagName=[8086ACE92F504F9BBF5D377FF2C5034C/BAF7EEC3B07E4206A3ABB2224017FC20] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-17 21:30:09,770 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8086ACE92F504F9BBF5D377FF2C5034C/BAF7EEC3B07E4206A3ABB2224017FC20] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-17 21:30:09,772 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520609631_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8086ACE92F504F9BBF5D377FF2C5034C/BAF7EEC3B07E4206A3ABB2224017FC20/.tez/application_1463520609631_0001/localmode-log-dir/dag_1463520609631_0001_1.dot
2016-05-17 21:30:09,772 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8086ACE92F504F9BBF5D377FF2C5034C/BAF7EEC3B07E4206A3ABB2224017FC20/.tez/application_1463520609631_0001/localmode-log-dir/dag_1463520609631_0001_1-tez-dag.pb.txt
2016-05-17 21:30:09,777 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520609631_0001_1
2016-05-17 21:30:09,777 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:09,777 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8086ACE92F504F9BBF5D377FF2C5034C/BAF7EEC3B07E4206A3ABB2224017FC20] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-17 21:30:09 Running Dag: dag_1463520609631_0001_1
2016-05-17 21:30:09,788 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609631_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520609631_0001_1, submitTime=1463520609770
2016-05-17 21:30:09,788 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:09,789 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E8CC9DF9522A455F89D9E787E266E923
2016-05-17 21:30:09,789 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:09,789 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609631_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520609631_0001_1, initTime=1463520609788
2016-05-17 21:30:09,789 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:09,790 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520609631_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:09,790 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:09,790 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609631_0001_1][Event:DAG_STARTED]: dagID=dag_1463520609631_0001_1, startTime=1463520609790
2016-05-17 21:30:09,791 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:09,791 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520609631_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:09,791 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...RESULTS.0_UNKNOWN_RESULTS, applicationId=application_1463520609631_0001, dagName=[8086ACE92F504F9BBF5D377FF2C5034C/BAF7EEC3B07E4206A3ABB2224017FC20] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-17 21:30:09,791 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E8CC9DF9522A455F89D9E787E266E923 : {4FF426756C5143688777293703EB168E={InputName=4FF426756C5143688777293703EB168E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:09,791 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4FF426756C5143688777293703EB168E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:09,791 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923]
2016-05-17 21:30:09,791 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923]
2016-05-17 21:30:09,791 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923]
2016-05-17 21:30:09,792 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923]. Starting root input initializers: 1
2016-05-17 21:30:09,792 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:09,792 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4FF426756C5143688777293703EB168E on vertex vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923]
2016-05-17 21:30:09,794 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:09,794 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4FF426756C5143688777293703EB168E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:09,794 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:09,795 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:09,796 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:09,796 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4FF426756C5143688777293703EB168E on vertex vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923]
2016-05-17 21:30:09,797 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:09,797 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4FF426756C5143688777293703EB168E=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:09,797 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923] parallelism set to 1
2016-05-17 21:30:09,797 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:09,797 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609631_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520609631_0001_1_00, eventCount=1
2016-05-17 21:30:09,797 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923]
2016-05-17 21:30:09,798 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=ECD9B3A6615E408596E06DBFBD5BC6A9, vertexId=vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:09,798 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=ECD9B3A6615E408596E06DBFBD5BC6A9, vertexId=vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923]
2016-05-17 21:30:09,799 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=ECD9B3A6615E408596E06DBFBD5BC6A9, vertexName=E8CC9DF9522A455F89D9E787E266E923, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:09,800 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=ECD9B3A6615E408596E06DBFBD5BC6A9, vertexId=vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923]
2016-05-17 21:30:09,801 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609631_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E8CC9DF9522A455F89D9E787E266E923, vertexId=vertex_1463520609631_0001_1_00, initRequestedTime=1463520609791, initedTime=1463520609800, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:09,801 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:09,801 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923] with distanceFromRoot: 0
2016-05-17 21:30:09,801 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:09,802 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E8CC9DF9522A455F89D9E787E266E923
2016-05-17 21:30:09,802 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923]
2016-05-17 21:30:09,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:09,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609631_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520609631_0001_1_00, startRequestedTime=1463520609802, startedTime=1463520609802
2016-05-17 21:30:09,802 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:09,803 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:09,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609631_0001_1][Event:TASK_STARTED]: vertexName=E8CC9DF9522A455F89D9E787E266E923, taskId=task_1463520609631_0001_1_00_000000, scheduledTime=1463520609803, launchTime=1463520609803
2016-05-17 21:30:09,803 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520609631_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:09,803 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923]
2016-05-17 21:30:09,804 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8086ACE92F504F9BBF5D377FF2C5034C/BAF7EEC3B07E4206A3ABB2224017FC20] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS, VertexName: E8CC9DF9522A455F89D9E787E266E923, VertexParallelism: 1, TaskAttemptID:attempt_1463520609631_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4FF426756C5143688777293703EB168E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=ECD9B3A6615E408596E06DBFBD5BC6A9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:09,804 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520609631_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:09,804 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:09,805 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:09,805 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:09,805 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:09,806 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520609631_0001_00_000001
2016-05-17 21:30:09,806 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520609631_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:09,807 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520609631_0001_00_000001
2016-05-17 21:30:09,807 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:09,808 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520609631_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:09,808 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:09,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609631_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520609631_0001_00_000001, launchTime=1463520609808
2016-05-17 21:30:09,809 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520609631_0001_00_000001
2016-05-17 21:30:09,811 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520609631_0001_1_00_000000_0] to container: [container_1463520609631_0001_00_000001]
2016-05-17 21:30:09,811 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520609631_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:09,812 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520609631_0001, with dag id: dag_1463520609631_0001_1
2016-05-17 21:30:09,812 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520609631_0001_1_00_000000_0] started. Is using containerId: [container_1463520609631_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:09,812 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609631_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E8CC9DF9522A455F89D9E787E266E923, taskAttemptId=attempt_1463520609631_0001_1_00_000000_0, startTime=1463520609812, containerId=container_1463520609631_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520609631_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:09,813 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520609631_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:09,812 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:09,813 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520609631_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:09,813 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520609631_0001_00_000001 given task: attempt_1463520609631_0001_1_00_000000_0
2016-05-17 21:30:09,813 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520609631_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520609631_0001_1_00_000000_0
2016-05-17 21:30:09 Starting to run new task attempt: attempt_1463520609631_0001_1_00_000000_0
2016-05-17 21:30:09,814 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520609631_0001_1_00_000000_0
2016-05-17 21:30:09,814 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:09,814 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:09,814 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:09,814 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:09,814 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:09,814 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:09,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8086ACE92F504F9BBF5D377FF2C5034C/BAF7EEC3B07E4206A3ABB2224017FC20] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS, VertexName: E8CC9DF9522A455F89D9E787E266E923, VertexParallelism: 1, TaskAttemptID:attempt_1463520609631_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4FF426756C5143688777293703EB168E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=ECD9B3A6615E408596E06DBFBD5BC6A9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:09,815 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:09,815 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520609631_0001_1_00_000000_0
2016-05-17 21:30:09,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:09,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:09,817 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:09,817 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520609631_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:09,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=ECD9B3A6615E408596E06DBFBD5BC6A9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:09,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:09,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: ECD9B3A6615E408596E06DBFBD5BC6A9
2016-05-17 21:30:09,819 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:09,819 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:09,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4FF426756C5143688777293703EB168E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:09,819 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: ECD9B3A6615E408596E06DBFBD5BC6A9
2016-05-17 21:30:09,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:09,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4FF426756C5143688777293703EB168E
2016-05-17 21:30:09,822 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4FF426756C5143688777293703EB168E
2016-05-17 21:30:09,828 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:09,828 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4FF426756C5143688777293703EB168E
2016-05-17 21:30:09,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4FF426756C5143688777293703EB168E
2016-05-17 21:30:09,832 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E8CC9DF9522A455F89D9E787E266E923, ordinal: 0
2016-05-17 21:30:09,832 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8CC9DF9522A455F89D9E787E266E923, mem on start (mb), free: 185, total: 381, max: 672
2016-05-17 21:30:09,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:09,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:09,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:09,835 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:09,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: ECD9B3A6615E408596E06DBFBD5BC6A9
2016-05-17 21:30:09,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:09,836 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:09,836 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:09,836 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:09,836 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:09,836 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:09,836 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:09,837 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:09,837 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, ECD9B3A6615E408596E06DBFBD5BC6A9, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:09,837 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4FF426756C5143688777293703EB168E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:09,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:09,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4FF426756C5143688777293703EB168E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:09,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:09,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:09,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4FF426756C5143688777293703EB168E
2016-05-17 21:30:09,839 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:09,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4FF426756C5143688777293703EB168E
2016-05-17 21:30:09,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:09,840 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520609631_0001_1_00_000000_0
2016-05-17 21:30:09,840 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:09,841 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3e5e5b61
2016-05-17 21:30:09,844 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3e5e5b61
2016-05-17 21:30:09,844 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:09,844 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:09,845 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:09,846 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 4FF426756C5143688777293703EB168E
2016-05-17 21:30:09,847 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS"], id: ECD9B3A6615E408596E06DBFBD5BC6A9
2016-05-17 21:30:09,848 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS"]
2016-05-17 21:30:09,848 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:09,848 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:09,849 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E8CC9DF9522A455F89D9E787E266E923, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:09,851 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:09,852 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:09,852 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520609631_0001_1_00_000000_0
2016-05-17 21:30:09,853 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520609631_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:09,853 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520609631_0001_1_00_000000_0
2016-05-17 21:30:09,854 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8CC9DF9522A455F89D9E787E266E923, mem on close (mb), free: 184, total: 381, max: 672
2016-05-17 21:30:09 Completed running task attempt: attempt_1463520609631_0001_1_00_000000_0
2016-05-17 21:30:09,854 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520609631_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:09,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520609849, Process_Duration=4, Process_End_Time=1463520609853, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:09,857 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:09,857 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:09,857 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:09,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609631_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E8CC9DF9522A455F89D9E787E266E923, taskAttemptId=attempt_1463520609631_0001_1_00_000000_0, startTime=1463520609812, finishTime=1463520609857, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520609849, Process_Duration=4, Process_End_Time=1463520609853, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:09,858 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520609631_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:09,858 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520609631_0001_1_00_000000_0
2016-05-17 21:30:09,858 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:09,858 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520609631_0001_00_000001
2016-05-17 21:30:09,859 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:09,859 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609631_0001_1][Event:TASK_FINISHED]: vertexName=E8CC9DF9522A455F89D9E787E266E923, taskId=task_1463520609631_0001_1_00_000000, startTime=1463520609812, finishTime=1463520609858, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1463520609631_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520609849, Process_Duration=4, Process_End_Time=1463520609853, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:09,860 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520609631_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:09,861 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923] : 1
2016-05-17 21:30:09,861 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923]
2016-05-17 21:30:09,862 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:09,862 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609631_0001_1][Event:VERTEX_FINISHED]: vertexName=E8CC9DF9522A455F89D9E787E266E923, vertexId=vertex_1463520609631_0001_1_00, initRequestedTime=1463520609791, initedTime=1463520609800, startRequestedTime=1463520609802, startedTime=1463520609802, finishTime=1463520609861, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520609849, Process_Duration=4, Process_End_Time=1463520609853, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520609812, firstTasksToStart=[ task_1463520609631_0001_1_00_000000 ], lastTaskFinishTime=1463520609857, lastTasksToFinish=[ task_1463520609631_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520609631_0001_1_00_000000 ], longestDurationTasks=[ task_1463520609631_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:09,863 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:09,863 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520609631_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:09,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609631_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520609631_0001_00_000001, stoppedTime=1463520609863, exitStatus=0
2016-05-17 21:30:09,863 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520609631_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:09,863 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:09,863 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520609631_0001_1
2016-05-17 21:30:09,864 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:09,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609631_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520609631_0001_1
2016-05-17 21:30:09,864 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: ECD9B3A6615E408596E06DBFBD5BC6A9 for vertex: vertex_1463520609631_0001_1_00 [E8CC9DF9522A455F89D9E787E266E923]
2016-05-17 21:30:09,869 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520609631_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:09,869 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520609631_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:09,869 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520609631_0001_00_000001, Exiting
2016-05-17 21:30:09,869 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520609631_0001_00_000001
2016-05-17 21:30:09,869 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520609631_0001_00_000001
2016-05-17 21:30:09,872 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520609631_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:09,873 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520609631_0001_00_000001 completed successfully
2016-05-17 21:30:09,875 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520609631_0001_1, queueSize=0
2016-05-17 21:30:09,876 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:09,876 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520609631_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520609631_0001_1, startTime=1463520609790, finishTime=1463520609874, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=399507456, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520609849, Process_Duration=4, Process_End_Time=1463520609853, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:09,876 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520609631_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:09,877 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520609631_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:09,877 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520609631_0001_00_000001
2016-05-17 21:30:09,878 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520609631_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:09 Completed Dag: dag_1463520609631_0001_1
2016-05-17 21:30:09,879 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520609631_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:09,880 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520609631_0001_1_post
2016-05-17 21:30:09,880 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:09,880 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:09,884 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...RESULTS.0_UNKNOWN_RESULTS, applicationId=application_1463520609631_0001
2016-05-17 21:30:09,886 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:09,886 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:09,886 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:09,886 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:09,887 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:09,887 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:09,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:09,888 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:09,888 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:09,888 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:09,888 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:09,888 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:09,888 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:09,892 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8086ACE92F504F9BBF5D377FF2C5034C/BAF7EEC3B07E4206A3ABB2224017FC20/.tez/application_1463520609631_0001
2016-05-17 21:30:09,892 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS/_temporary
2016-05-17 21:30:09,894 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.281
2016-05-17 21:30:09,961 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_RESULTS
2016-05-17 21:30:09,980 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:09,981 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:10,011 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:10,011 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:10,016 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 21:30:10,022 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 21:30:10,022 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:10,022 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:10,023 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:10,036 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:10,037 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL
2016-05-17 21:30:10,046 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:10,046 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:10,046 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL"]
2016-05-17 21:30:10,046 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:10,046 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:10,046 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:10,048 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-17 21:30:10,048 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:10,060 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:10,065 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:10,065 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:10,065 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:10,074 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/9D4F1F04098C46F9BDC8937AF9C79FBE/C579B00DE12D4D5F907DAEB9A18B844F/.tez/application_1463520610065_0001 doesn't exist and is created
2016-05-17 21:30:10,089 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:10,090 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:10,090 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/9D4F1F04098C46F9BDC8937AF9C79FBE/C579B00DE12D4D5F907DAEB9A18B844F/.tez/application_1463520610065_0001
2016-05-17 21:30:10,093 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520610065_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:10,103 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:10,104 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:10,105 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:10,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:10,106 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/9D4F1F04098C46F9BDC8937AF9C79FBE/C579B00DE12D4D5F907DAEB9A18B844F/.tez/application_1463520610065_0001/localmode-log-dir
2016-05-17 21:30:10,106 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/9D4F1F04098C46F9BDC8937AF9C79FBE/C579B00DE12D4D5F907DAEB9A18B844F/.tez/application_1463520610065_0001/localmode-log-dir/history.txt.appattempt_1463520610065_0001_000000, maxErrors=10
2016-05-17 21:30:10,106 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:10,107 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520610065_0001_000000, appSubmitTime=1463520610092, launchTime=1463520610093
2016-05-17 21:30:10,112 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:10,116 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:10,116 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:52686
2016-05-17 21:30:10,117 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:10,117 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:10,118 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:10,122 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520610065_0001_000000, startTime=1463520610122
2016-05-17 21:30:10,122 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:10,122 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:10,190 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:10,190 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:10,190 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, applicationId=application_1463520610065_0001, dagName=[9D4F1F04098C46F9BDC8937AF9C79FBE/C579B00DE12D4D5F907DAEB9A18B844F] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-17 21:30:10,204 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [9D4F1F04098C46F9BDC8937AF9C79FBE/C579B00DE12D4D5F907DAEB9A18B844F] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-17 21:30:10,205 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520610065_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/9D4F1F04098C46F9BDC8937AF9C79FBE/C579B00DE12D4D5F907DAEB9A18B844F/.tez/application_1463520610065_0001/localmode-log-dir/dag_1463520610065_0001_1.dot
2016-05-17 21:30:10,206 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/9D4F1F04098C46F9BDC8937AF9C79FBE/C579B00DE12D4D5F907DAEB9A18B844F/.tez/application_1463520610065_0001/localmode-log-dir/dag_1463520610065_0001_1-tez-dag.pb.txt
2016-05-17 21:30:10,224 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520610065_0001_1
2016-05-17 21:30:10,224 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:10,224 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [9D4F1F04098C46F9BDC8937AF9C79FBE/C579B00DE12D4D5F907DAEB9A18B844F] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-17 21:30:10 Running Dag: dag_1463520610065_0001_1
2016-05-17 21:30:10,235 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610065_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520610065_0001_1, submitTime=1463520610204
2016-05-17 21:30:10,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:10,235 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DDAB7B925C254516A50A322C5A8EE8DB
2016-05-17 21:30:10,236 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:10,236 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610065_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520610065_0001_1, initTime=1463520610235
2016-05-17 21:30:10,236 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:10,237 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520610065_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:10,238 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610065_0001_1][Event:DAG_STARTED]: dagID=dag_1463520610065_0001_1, startTime=1463520610237
2016-05-17 21:30:10,238 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:10,238 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520610065_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:10,238 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DDAB7B925C254516A50A322C5A8EE8DB : {02C62A74B2EC4B05815136D018E9226C={InputName=02C62A74B2EC4B05815136D018E9226C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:10,238 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 02C62A74B2EC4B05815136D018E9226C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:10,238 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB]
2016-05-17 21:30:10,238 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB]
2016-05-17 21:30:10,239 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB]
2016-05-17 21:30:10,239 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB]. Starting root input initializers: 1
2016-05-17 21:30:10,239 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:10,239 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:10,240 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 02C62A74B2EC4B05815136D018E9226C on vertex vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB]
2016-05-17 21:30:10,240 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, applicationId=application_1463520610065_0001, dagName=[9D4F1F04098C46F9BDC8937AF9C79FBE/C579B00DE12D4D5F907DAEB9A18B844F] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-17 21:30:10,241 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:10,241 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 02C62A74B2EC4B05815136D018E9226C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:10,242 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:10,242 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:10,243 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:10,243 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 02C62A74B2EC4B05815136D018E9226C on vertex vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB]
2016-05-17 21:30:10,243 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:10,243 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {02C62A74B2EC4B05815136D018E9226C=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:10,244 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB] parallelism set to 1
2016-05-17 21:30:10,244 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:10,244 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610065_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520610065_0001_1_00, eventCount=1
2016-05-17 21:30:10,244 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB]
2016-05-17 21:30:10,244 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=062157BBAB6847C0A4302093BC12DAAF, vertexId=vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:10,244 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=062157BBAB6847C0A4302093BC12DAAF, vertexId=vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB]
2016-05-17 21:30:10,246 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=062157BBAB6847C0A4302093BC12DAAF, vertexName=DDAB7B925C254516A50A322C5A8EE8DB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:10,246 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=062157BBAB6847C0A4302093BC12DAAF, vertexId=vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB]
2016-05-17 21:30:10,247 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610065_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DDAB7B925C254516A50A322C5A8EE8DB, vertexId=vertex_1463520610065_0001_1_00, initRequestedTime=1463520610238, initedTime=1463520610247, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:10,247 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:10,247 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB] with distanceFromRoot: 0
2016-05-17 21:30:10,247 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:10,247 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DDAB7B925C254516A50A322C5A8EE8DB
2016-05-17 21:30:10,247 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB]
2016-05-17 21:30:10,248 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:10,248 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610065_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520610065_0001_1_00, startRequestedTime=1463520610247, startedTime=1463520610247
2016-05-17 21:30:10,248 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:10,248 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:10,249 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610065_0001_1][Event:TASK_STARTED]: vertexName=DDAB7B925C254516A50A322C5A8EE8DB, taskId=task_1463520610065_0001_1_00_000000, scheduledTime=1463520610248, launchTime=1463520610248
2016-05-17 21:30:10,249 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520610065_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:10,249 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB]
2016-05-17 21:30:10,249 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [9D4F1F04098C46F9BDC8937AF9C79FBE/C579B00DE12D4D5F907DAEB9A18B844F] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, VertexName: DDAB7B925C254516A50A322C5A8EE8DB, VertexParallelism: 1, TaskAttemptID:attempt_1463520610065_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=02C62A74B2EC4B05815136D018E9226C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=062157BBAB6847C0A4302093BC12DAAF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:10,249 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520610065_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:10,250 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:10,250 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:10,250 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:10,251 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:10,251 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520610065_0001_00_000001
2016-05-17 21:30:10,251 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610065_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:10,253 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520610065_0001_00_000001
2016-05-17 21:30:10,253 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:10,254 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610065_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:10,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:10,254 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610065_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520610065_0001_00_000001, launchTime=1463520610254
2016-05-17 21:30:10,255 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520610065_0001_00_000001
2016-05-17 21:30:10,255 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520610065_0001_1_00_000000_0] to container: [container_1463520610065_0001_00_000001]
2016-05-17 21:30:10,256 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610065_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:10,256 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520610065_0001_00_000001 given task: attempt_1463520610065_0001_1_00_000000_0
2016-05-17 21:30:10,256 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520610065_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520610065_0001_1_00_000000_0
2016-05-17 21:30:10,256 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520610065_0001_1_00_000000_0] started. Is using containerId: [container_1463520610065_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:10 Starting to run new task attempt: attempt_1463520610065_0001_1_00_000000_0
2016-05-17 21:30:10,256 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610065_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DDAB7B925C254516A50A322C5A8EE8DB, taskAttemptId=attempt_1463520610065_0001_1_00_000000_0, startTime=1463520610256, containerId=container_1463520610065_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520610065_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:10,256 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520610065_0001_1_00_000000_0
2016-05-17 21:30:10,257 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:10,257 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520610065_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:10,257 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:10,257 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520610065_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:10,257 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:10,257 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:10,257 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:10,257 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:10,258 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:10,258 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [9D4F1F04098C46F9BDC8937AF9C79FBE/C579B00DE12D4D5F907DAEB9A18B844F] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, VertexName: DDAB7B925C254516A50A322C5A8EE8DB, VertexParallelism: 1, TaskAttemptID:attempt_1463520610065_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=02C62A74B2EC4B05815136D018E9226C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=062157BBAB6847C0A4302093BC12DAAF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:10,258 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:10,259 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520610065_0001_1_00_000000_0
2016-05-17 21:30:10,264 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:10,264 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520610065_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:10,264 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:10,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:10,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=02C62A74B2EC4B05815136D018E9226C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:10,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:10,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 02C62A74B2EC4B05815136D018E9226C
2016-05-17 21:30:10,266 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 02C62A74B2EC4B05815136D018E9226C
2016-05-17 21:30:10,267 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=062157BBAB6847C0A4302093BC12DAAF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:10,268 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:10,267 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:10,269 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 02C62A74B2EC4B05815136D018E9226C
2016-05-17 21:30:10,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 02C62A74B2EC4B05815136D018E9226C
2016-05-17 21:30:10,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:10,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 062157BBAB6847C0A4302093BC12DAAF
2016-05-17 21:30:10,270 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:10,270 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520610065_0001, with dag id: dag_1463520610065_0001_1
2016-05-17 21:30:10,271 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 062157BBAB6847C0A4302093BC12DAAF
2016-05-17 21:30:10,282 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DDAB7B925C254516A50A322C5A8EE8DB, ordinal: 0
2016-05-17 21:30:10,283 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DDAB7B925C254516A50A322C5A8EE8DB, mem on start (mb), free: 309, total: 382, max: 672
2016-05-17 21:30:10,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:10,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:10,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:10,285 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:10,285 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 062157BBAB6847C0A4302093BC12DAAF
2016-05-17 21:30:10,285 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:10,285 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:10,286 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:10,286 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:10,286 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:10,286 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:10,286 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:10,286 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:10,287 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 02C62A74B2EC4B05815136D018E9226C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:10,287 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 062157BBAB6847C0A4302093BC12DAAF, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:10,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:10,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 02C62A74B2EC4B05815136D018E9226C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:10,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 02C62A74B2EC4B05815136D018E9226C
2016-05-17 21:30:10,287 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:10,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 02C62A74B2EC4B05815136D018E9226C
2016-05-17 21:30:10,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:10,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:10,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:10,288 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520610065_0001_1_00_000000_0
2016-05-17 21:30:10,289 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:10,290 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@304da31b
2016-05-17 21:30:10,291 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@304da31b
2016-05-17 21:30:10,292 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:10,292 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:10,294 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:10,294 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 02C62A74B2EC4B05815136D018E9226C
2016-05-17 21:30:10,295 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL"], id: 062157BBAB6847C0A4302093BC12DAAF
2016-05-17 21:30:10,295 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL"]
2016-05-17 21:30:10,295 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:10,296 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:10,296 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DDAB7B925C254516A50A322C5A8EE8DB, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:10,299 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:10,299 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:10,299 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520610065_0001_1_00_000000_0
2016-05-17 21:30:10,300 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520610065_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:10,300 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520610065_0001_1_00_000000_0
2016-05-17 21:30:10,301 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DDAB7B925C254516A50A322C5A8EE8DB, mem on close (mb), free: 307, total: 382, max: 672
2016-05-17 21:30:10 Completed running task attempt: attempt_1463520610065_0001_1_00_000000_0
2016-05-17 21:30:10,302 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520610065_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:10,304 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=792, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520610296, Process_Duration=4, Process_End_Time=1463520610300, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:10,304 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:10,304 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610065_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DDAB7B925C254516A50A322C5A8EE8DB, taskAttemptId=attempt_1463520610065_0001_1_00_000000_0, startTime=1463520610256, finishTime=1463520610304, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=792, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520610296, Process_Duration=4, Process_End_Time=1463520610300, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:10,304 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520610065_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:10,304 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520610065_0001_1_00_000000_0
2016-05-17 21:30:10,305 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:10,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610065_0001_1][Event:TASK_FINISHED]: vertexName=DDAB7B925C254516A50A322C5A8EE8DB, taskId=task_1463520610065_0001_1_00_000000, startTime=1463520610256, finishTime=1463520610304, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1463520610065_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=792, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520610296, Process_Duration=4, Process_End_Time=1463520610300, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:10,305 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520610065_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:10,305 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB] : 1
2016-05-17 21:30:10,305 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB]
2016-05-17 21:30:10,306 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:10,306 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:10,309 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:10,309 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610065_0001_1][Event:VERTEX_FINISHED]: vertexName=DDAB7B925C254516A50A322C5A8EE8DB, vertexId=vertex_1463520610065_0001_1_00, initRequestedTime=1463520610238, initedTime=1463520610247, startRequestedTime=1463520610247, startedTime=1463520610247, finishTime=1463520610305, timeTaken=58, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=792, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520610296, Process_Duration=4, Process_End_Time=1463520610300, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520610256, firstTasksToStart=[ task_1463520610065_0001_1_00_000000 ], lastTaskFinishTime=1463520610304, lastTasksToFinish=[ task_1463520610065_0001_1_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520610065_0001_1_00_000000 ], longestDurationTasks=[ task_1463520610065_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:10,310 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:10,309 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:10,310 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610065_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:10,310 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610065_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520610065_0001_00_000001, stoppedTime=1463520610310, exitStatus=0
2016-05-17 21:30:10,310 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610065_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:10,310 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:10,311 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520610065_0001_1
2016-05-17 21:30:10,310 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520610065_0001_00_000001
2016-05-17 21:30:10,311 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:10,311 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610065_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520610065_0001_1
2016-05-17 21:30:10,311 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 062157BBAB6847C0A4302093BC12DAAF for vertex: vertex_1463520610065_0001_1_00 [DDAB7B925C254516A50A322C5A8EE8DB]
2016-05-17 21:30:10,312 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520610065_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:10,313 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520610065_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:10,313 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520610065_0001_00_000001, Exiting
2016-05-17 21:30:10,313 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520610065_0001_00_000001
2016-05-17 21:30:10,313 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520610065_0001_00_000001
2016-05-17 21:30:10,314 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520610065_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:10,314 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520610065_0001_00_000001 completed successfully
2016-05-17 21:30:10,321 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520610065_0001_1, queueSize=0
2016-05-17 21:30:10,321 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:10,321 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610065_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520610065_0001_1, startTime=1463520610237, finishTime=1463520610319, timeTaken=82, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=792, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520610296, Process_Duration=4, Process_End_Time=1463520610300, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:10,321 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520610065_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:10,321 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520610065_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:10,321 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520610065_0001_00_000001
2016-05-17 21:30:10,321 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610065_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:10 Completed Dag: dag_1463520610065_0001_1
2016-05-17 21:30:10,322 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520610065_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:10,322 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520610065_0001_1_post
2016-05-17 21:30:10,322 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:10,322 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:10,328 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, applicationId=application_1463520610065_0001
2016-05-17 21:30:10,331 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:10,331 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:10,331 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:10,331 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:10,331 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:10,332 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:10,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:10,332 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:10,332 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:10,332 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:10,333 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:10,333 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:10,334 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:10,340 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/9D4F1F04098C46F9BDC8937AF9C79FBE/C579B00DE12D4D5F907DAEB9A18B844F/.tez/application_1463520610065_0001
2016-05-17 21:30:10,340 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL/_temporary
2016-05-17 21:30:10,341 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.294
2016-05-17 21:30:10,407 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_ALL
2016-05-17 21:30:10,426 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:10,426 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:10,455 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 21:30:10,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 21:30:10,463 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:10,463 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:10,463 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:10,475 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:10,476 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1
2016-05-17 21:30:10,486 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:10,486 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:10,486 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1"]
2016-05-17 21:30:10,486 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:10,486 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:10,486 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:10,487 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-17 21:30:10,487 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:10,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:10,503 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:10,503 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:10,503 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:10,512 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8D56B2F40AF343708D0021DFB5D88AA0/C7B429A83A514F3BBFA3596BD69B1B8A/.tez/application_1463520610503_0001 doesn't exist and is created
2016-05-17 21:30:10,528 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:10,528 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:10,529 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8D56B2F40AF343708D0021DFB5D88AA0/C7B429A83A514F3BBFA3596BD69B1B8A/.tez/application_1463520610503_0001
2016-05-17 21:30:10,532 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520610503_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:10,534 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:10,534 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:10,543 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:10,544 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:10,545 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:10,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:10,546 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8D56B2F40AF343708D0021DFB5D88AA0/C7B429A83A514F3BBFA3596BD69B1B8A/.tez/application_1463520610503_0001/localmode-log-dir
2016-05-17 21:30:10,546 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8D56B2F40AF343708D0021DFB5D88AA0/C7B429A83A514F3BBFA3596BD69B1B8A/.tez/application_1463520610503_0001/localmode-log-dir/history.txt.appattempt_1463520610503_0001_000000, maxErrors=10
2016-05-17 21:30:10,546 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:10,546 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520610503_0001_000000, appSubmitTime=1463520610531, launchTime=1463520610531
2016-05-17 21:30:10,552 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:10,553 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43676
2016-05-17 21:30:10,553 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:10,554 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:10,555 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:10,556 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:10,557 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520610503_0001_000000, startTime=1463520610557
2016-05-17 21:30:10,557 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:10,557 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:10,628 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:10,629 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:10,629 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ield'_RESULTS.0_VALUES_-1, applicationId=application_1463520610503_0001, dagName=[8D56B2F40AF343708D0021DFB5D88AA0/C7B429A83A514F3BBFA3596BD69B1B8A] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-17 21:30:10,643 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8D56B2F40AF343708D0021DFB5D88AA0/C7B429A83A514F3BBFA3596BD69B1B8A] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-17 21:30:10,644 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520610503_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8D56B2F40AF343708D0021DFB5D88AA0/C7B429A83A514F3BBFA3596BD69B1B8A/.tez/application_1463520610503_0001/localmode-log-dir/dag_1463520610503_0001_1.dot
2016-05-17 21:30:10,645 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8D56B2F40AF343708D0021DFB5D88AA0/C7B429A83A514F3BBFA3596BD69B1B8A/.tez/application_1463520610503_0001/localmode-log-dir/dag_1463520610503_0001_1-tez-dag.pb.txt
2016-05-17 21:30:10,649 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520610503_0001_1
2016-05-17 21:30:10,649 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:10,649 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8D56B2F40AF343708D0021DFB5D88AA0/C7B429A83A514F3BBFA3596BD69B1B8A] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-17 21:30:10 Running Dag: dag_1463520610503_0001_1
2016-05-17 21:30:10,660 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610503_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520610503_0001_1, submitTime=1463520610643
2016-05-17 21:30:10,660 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:10,661 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F9AB1D8F81FA4FA8B9BB2BB50F6777D2
2016-05-17 21:30:10,661 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:10,661 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610503_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520610503_0001_1, initTime=1463520610660
2016-05-17 21:30:10,661 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:10,661 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520610503_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:10,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610503_0001_1][Event:DAG_STARTED]: dagID=dag_1463520610503_0001_1, startTime=1463520610662
2016-05-17 21:30:10,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:10,662 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:10,663 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520610503_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:10,663 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F9AB1D8F81FA4FA8B9BB2BB50F6777D2 : {47B573247FB14CFA90FE9B8FF7264B08={InputName=47B573247FB14CFA90FE9B8FF7264B08}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:10,663 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 47B573247FB14CFA90FE9B8FF7264B08, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:10,663 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2]
2016-05-17 21:30:10,663 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2]
2016-05-17 21:30:10,663 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2]
2016-05-17 21:30:10,664 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2]. Starting root input initializers: 1
2016-05-17 21:30:10,664 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ield'_RESULTS.0_VALUES_-1, applicationId=application_1463520610503_0001, dagName=[8D56B2F40AF343708D0021DFB5D88AA0/C7B429A83A514F3BBFA3596BD69B1B8A] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-17 21:30:10,664 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:10,665 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 47B573247FB14CFA90FE9B8FF7264B08 on vertex vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2]
2016-05-17 21:30:10,667 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:10,667 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 47B573247FB14CFA90FE9B8FF7264B08 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:10,667 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:10,667 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:10,669 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:10,669 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 47B573247FB14CFA90FE9B8FF7264B08 on vertex vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2]
2016-05-17 21:30:10,669 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:10,669 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {47B573247FB14CFA90FE9B8FF7264B08=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:10,669 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2] parallelism set to 1
2016-05-17 21:30:10,670 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:10,670 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610503_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520610503_0001_1_00, eventCount=1
2016-05-17 21:30:10,670 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2]
2016-05-17 21:30:10,670 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=BD34191F37B4416EA55F1B77A4447543, vertexId=vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:10,670 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=BD34191F37B4416EA55F1B77A4447543, vertexId=vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2]
2016-05-17 21:30:10,672 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=BD34191F37B4416EA55F1B77A4447543, vertexName=F9AB1D8F81FA4FA8B9BB2BB50F6777D2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:10,672 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=BD34191F37B4416EA55F1B77A4447543, vertexId=vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2]
2016-05-17 21:30:10,673 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:10,673 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610503_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F9AB1D8F81FA4FA8B9BB2BB50F6777D2, vertexId=vertex_1463520610503_0001_1_00, initRequestedTime=1463520610663, initedTime=1463520610673, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:10,673 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2] with distanceFromRoot: 0
2016-05-17 21:30:10,673 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:10,673 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F9AB1D8F81FA4FA8B9BB2BB50F6777D2
2016-05-17 21:30:10,674 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2]
2016-05-17 21:30:10,674 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:10,674 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610503_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520610503_0001_1_00, startRequestedTime=1463520610673, startedTime=1463520610673
2016-05-17 21:30:10,674 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:10,674 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:10,675 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610503_0001_1][Event:TASK_STARTED]: vertexName=F9AB1D8F81FA4FA8B9BB2BB50F6777D2, taskId=task_1463520610503_0001_1_00_000000, scheduledTime=1463520610674, launchTime=1463520610674
2016-05-17 21:30:10,675 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520610503_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:10,675 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2]
2016-05-17 21:30:10,675 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8D56B2F40AF343708D0021DFB5D88AA0/C7B429A83A514F3BBFA3596BD69B1B8A] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1, VertexName: F9AB1D8F81FA4FA8B9BB2BB50F6777D2, VertexParallelism: 1, TaskAttemptID:attempt_1463520610503_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=47B573247FB14CFA90FE9B8FF7264B08, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BD34191F37B4416EA55F1B77A4447543, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:10,676 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520610503_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:10,676 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:10,676 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:10,677 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:10,677 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:10,677 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520610503_0001_00_000001
2016-05-17 21:30:10,678 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610503_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:10,680 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520610503_0001_00_000001
2016-05-17 21:30:10,680 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:10,681 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610503_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:10,681 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:10,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610503_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520610503_0001_00_000001, launchTime=1463520610681
2016-05-17 21:30:10,682 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520610503_0001_00_000001
2016-05-17 21:30:10,683 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520610503_0001_1_00_000000_0] to container: [container_1463520610503_0001_00_000001]
2016-05-17 21:30:10,684 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610503_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:10,685 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520610503_0001_1_00_000000_0] started. Is using containerId: [container_1463520610503_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:10,685 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610503_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F9AB1D8F81FA4FA8B9BB2BB50F6777D2, taskAttemptId=attempt_1463520610503_0001_1_00_000000_0, startTime=1463520610684, containerId=container_1463520610503_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520610503_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:10,685 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520610503_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:10,685 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520610503_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:10,685 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:10,686 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520610503_0001_00_000001 given task: attempt_1463520610503_0001_1_00_000000_0
2016-05-17 21:30:10,687 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520610503_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520610503_0001_1_00_000000_0
2016-05-17 21:30:10 Starting to run new task attempt: attempt_1463520610503_0001_1_00_000000_0
2016-05-17 21:30:10,683 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520610503_0001, with dag id: dag_1463520610503_0001_1
2016-05-17 21:30:10,687 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520610503_0001_1_00_000000_0
2016-05-17 21:30:10,688 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:10,688 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:10,688 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:10,689 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:10,689 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:10,689 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:10,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8D56B2F40AF343708D0021DFB5D88AA0/C7B429A83A514F3BBFA3596BD69B1B8A] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1, VertexName: F9AB1D8F81FA4FA8B9BB2BB50F6777D2, VertexParallelism: 1, TaskAttemptID:attempt_1463520610503_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=47B573247FB14CFA90FE9B8FF7264B08, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BD34191F37B4416EA55F1B77A4447543, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:10,689 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:10,690 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520610503_0001_1_00_000000_0
2016-05-17 21:30:10,690 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:10,690 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:10,691 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520610503_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:10,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=47B573247FB14CFA90FE9B8FF7264B08, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:10,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:10,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:10,693 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 47B573247FB14CFA90FE9B8FF7264B08
2016-05-17 21:30:10,693 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 47B573247FB14CFA90FE9B8FF7264B08
2016-05-17 21:30:10,695 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:10,695 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:10,696 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 47B573247FB14CFA90FE9B8FF7264B08
2016-05-17 21:30:10,696 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 47B573247FB14CFA90FE9B8FF7264B08
2016-05-17 21:30:10,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BD34191F37B4416EA55F1B77A4447543, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:10,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:10,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BD34191F37B4416EA55F1B77A4447543
2016-05-17 21:30:10,697 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:10,698 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: BD34191F37B4416EA55F1B77A4447543
2016-05-17 21:30:10,704 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F9AB1D8F81FA4FA8B9BB2BB50F6777D2, ordinal: 0
2016-05-17 21:30:10,704 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9AB1D8F81FA4FA8B9BB2BB50F6777D2, mem on start (mb), free: 235, total: 382, max: 672
2016-05-17 21:30:10,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:10,705 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:10,705 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:10,707 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:10,707 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BD34191F37B4416EA55F1B77A4447543
2016-05-17 21:30:10,707 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:10,708 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:10,708 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:10,708 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:10,708 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:10,708 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:10,708 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:10,708 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:10,709 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 47B573247FB14CFA90FE9B8FF7264B08, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:10,709 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BD34191F37B4416EA55F1B77A4447543, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:10,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:10,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 47B573247FB14CFA90FE9B8FF7264B08 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:10,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 47B573247FB14CFA90FE9B8FF7264B08
2016-05-17 21:30:10,709 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:10,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 47B573247FB14CFA90FE9B8FF7264B08
2016-05-17 21:30:10,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:10,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:10,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:10,710 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520610503_0001_1_00_000000_0
2016-05-17 21:30:10,710 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:10,713 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:10,714 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 47B573247FB14CFA90FE9B8FF7264B08
2016-05-17 21:30:10,714 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1"], id: BD34191F37B4416EA55F1B77A4447543
2016-05-17 21:30:10,715 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1"]
2016-05-17 21:30:10,715 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3e84ba1b
2016-05-17 21:30:10,715 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:10,716 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:10,716 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3e84ba1b
2016-05-17 21:30:10,717 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:10,717 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:10,717 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F9AB1D8F81FA4FA8B9BB2BB50F6777D2, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:10,718 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:10,721 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:10,722 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:10,722 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:10,723 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:10,723 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520610503_0001_1_00_000000_0
2016-05-17 21:30:10,723 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520610503_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:10,724 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520610503_0001_1_00_000000_0
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:10,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463520610717, Process_Duration=7, Process_End_Time=1463520610724, Read_Duration=1, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 21:30:10,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610503_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F9AB1D8F81FA4FA8B9BB2BB50F6777D2, taskAttemptId=attempt_1463520610503_0001_1_00_000000_0, startTime=1463520610684, finishTime=1463520610737, timeTaken=53, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520610717, Process_Duration=7, Process_End_Time=1463520610724, Read_Duration=1, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 21:30:10,738 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520610503_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:30:10,738 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520610503_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:30:10,738 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610503_0001_1][Event:TASK_FINISHED]: vertexName=F9AB1D8F81FA4FA8B9BB2BB50F6777D2, taskId=task_1463520610503_0001_1_00_000000, startTime=1463520610684, finishTime=1463520610738, timeTaken=54, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
], counters=Counters: 0
2016-05-17 21:30:10,739 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520610503_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:30:10,739 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2] : 1
2016-05-17 21:30:10,739 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2] because task failed: task_1463520610503_0001_1_00_000000
2016-05-17 21:30:10,739 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:30:10,739 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:30:10,739 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2]
2016-05-17 21:30:10,739 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=BD34191F37B4416EA55F1B77A4447543, vertexId=vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2]
2016-05-17 21:30:10,740 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:10,737 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:10,741 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:10,741 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610503_0001_1][Event:VERTEX_FINISHED]: vertexName=F9AB1D8F81FA4FA8B9BB2BB50F6777D2, vertexId=vertex_1463520610503_0001_1_00, initRequestedTime=1463520610663, initedTime=1463520610673, startRequestedTime=1463520610673, startedTime=1463520610673, finishTime=1463520610740, timeTaken=67, status=FAILED, diagnostics=Task failed, taskId=task_1463520610503_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520610684, firstTasksToStart=[ task_1463520610503_0001_1_00_000000 ], lastTaskFinishTime=1463520610737, lastTasksToFinish=[ task_1463520610503_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:30:10,741 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:30:10,741 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:10,742 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 21:30:10,742 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:30:10,742 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520610503_0001_1
2016-05-17 21:30:10,742 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: BD34191F37B4416EA55F1B77A4447543 for vertex: vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2]
2016-05-17 21:30:10,744 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9AB1D8F81FA4FA8B9BB2BB50F6777D2, mem on close (mb), free: 232, total: 382, max: 672
2016-05-17 21:30:10,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=F9AB1D8F81FA4FA8B9BB2BB50F6777D2, index=0
2016-05-17 21:30:10,744 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520610503_0001_1, queueSize=0
2016-05-17 21:30:10,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=F9AB1D8F81FA4FA8B9BB2BB50F6777D2, sourceVertex=47B573247FB14CFA90FE9B8FF7264B08
2016-05-17 21:30:10,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=F9AB1D8F81FA4FA8B9BB2BB50F6777D2, sourceVertex=BD34191F37B4416EA55F1B77A4447543
2016-05-17 21:30:10,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610503_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520610503_0001_1, startTime=1463520610662, finishTime=1463520610743, timeTaken=81, status=FAILED, diagnostics=Vertex failed, vertexName=F9AB1D8F81FA4FA8B9BB2BB50F6777D2, vertexId=vertex_1463520610503_0001_1_00, diagnostics=[Task failed, taskId=task_1463520610503_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520610503_0001_1_00 [F9AB1D8F81FA4FA8B9BB2BB50F6777D2] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 21:30:10,744 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520610503_0001_00_000001
2016-05-17 21:30:10,744 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520610503_0001_00_000001
2016-05-17 21:30:10,744 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:10,744 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520610503_0001_1 finished with state: FAILED
2016-05-17 21:30:10,746 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520610503_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:10 Completed Dag: dag_1463520610503_0001_1
2016-05-17 21:30:10,747 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520610503_0001_1, dagState=FAILED
2016-05-17 21:30:10,747 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520610503_0001_1_post
2016-05-17 21:30:10,747 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:10,747 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:10,748 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:10,748 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610503_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520610503_0001_00_000001, stoppedTime=1463520610748, exitStatus=0
2016-05-17 21:30:10,748 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520610503_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:30:10,748 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610503_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:10,748 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520610503_0001_1_00_000000_0 failed: true container: container_1463520610503_0001_00_000001 numFailedTAs: 0
2016-05-17 21:30:10,749 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520610503_0001_00_000001 completed but with errors
2016-05-17 21:30:10,749 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520610503_0001_00_000001
2016-05-17 21:30:10,749 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610503_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:30:10,749 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520610503_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:30:10,750 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610503_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:30:10,750 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520610503_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:30:10,762 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ield'_RESULTS.0_VALUES_-1, applicationId=application_1463520610503_0001
2016-05-17 21:30:10,764 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:10,764 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:10,764 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:10,764 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:10,764 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:10,765 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:10,765 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:10,766 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:10,766 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:10,766 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:10,766 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:10,766 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:10,766 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:10,773 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8D56B2F40AF343708D0021DFB5D88AA0/C7B429A83A514F3BBFA3596BD69B1B8A/.tez/application_1463520610503_0001
2016-05-17 21:30:10,773 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-17 21:30:10,774 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-17 21:30:10,774 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-17 21:30:10,775 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1/_temporary
2016-05-17 21:30:10,776 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.289
2016-05-17 21:30:10,776 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_-1
2016-05-17 21:30:10,797 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:10,797 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:10,823 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-17 21:30:10,827 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-17 21:30:10,828 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:10,828 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:10,828 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_'field2'
2016-05-17 21:30:10,858 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:10,858 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:10,900 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 21:30:10,906 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 21:30:10,907 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:10,907 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:10,907 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:10,928 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:10,929 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS
2016-05-17 21:30:10,943 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:10,943 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:10,944 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS"]
2016-05-17 21:30:10,944 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:10,944 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:10,944 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:10,947 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-17 21:30:10,947 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:10,965 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:10,972 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:10,972 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:10,972 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:10,984 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A42D8DE156304B5D822423C2E093633E/CF6BD2B209EB4F54875173E3FF0B1422/.tez/application_1463520610972_0001 doesn't exist and is created
2016-05-17 21:30:11,006 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:11,006 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:11,008 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A42D8DE156304B5D822423C2E093633E/CF6BD2B209EB4F54875173E3FF0B1422/.tez/application_1463520610972_0001
2016-05-17 21:30:11,012 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520610972_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:11,028 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:11,029 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:11,032 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:11,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:11,033 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A42D8DE156304B5D822423C2E093633E/CF6BD2B209EB4F54875173E3FF0B1422/.tez/application_1463520610972_0001/localmode-log-dir
2016-05-17 21:30:11,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A42D8DE156304B5D822423C2E093633E/CF6BD2B209EB4F54875173E3FF0B1422/.tez/application_1463520610972_0001/localmode-log-dir/history.txt.appattempt_1463520610972_0001_000000, maxErrors=10
2016-05-17 21:30:11,034 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:11,036 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520610972_0001_000000, appSubmitTime=1463520611011, launchTime=1463520611012
2016-05-17 21:30:11,042 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:11,044 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:11,045 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:11,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:11,049 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:39800
2016-05-17 21:30:11,051 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:11,053 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520610972_0001_000000, startTime=1463520611053
2016-05-17 21:30:11,053 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:11,053 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:11,061 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:11,061 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:11,107 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:11,107 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:11,107 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ..._RESULTS.0_VALUES_RESULTS, applicationId=application_1463520610972_0001, dagName=[A42D8DE156304B5D822423C2E093633E/CF6BD2B209EB4F54875173E3FF0B1422] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-17 21:30:11,128 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A42D8DE156304B5D822423C2E093633E/CF6BD2B209EB4F54875173E3FF0B1422] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-17 21:30:11,129 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520610972_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A42D8DE156304B5D822423C2E093633E/CF6BD2B209EB4F54875173E3FF0B1422/.tez/application_1463520610972_0001/localmode-log-dir/dag_1463520610972_0001_1.dot
2016-05-17 21:30:11,129 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A42D8DE156304B5D822423C2E093633E/CF6BD2B209EB4F54875173E3FF0B1422/.tez/application_1463520610972_0001/localmode-log-dir/dag_1463520610972_0001_1-tez-dag.pb.txt
2016-05-17 21:30:11,134 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520610972_0001_1
2016-05-17 21:30:11,134 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:11,134 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A42D8DE156304B5D822423C2E093633E/CF6BD2B209EB4F54875173E3FF0B1422] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-17 21:30:11 Running Dag: dag_1463520610972_0001_1
2016-05-17 21:30:11,145 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610972_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520610972_0001_1, submitTime=1463520611128
2016-05-17 21:30:11,146 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:11,146 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DF3E4B1AA1B445D081049E911A288CB5
2016-05-17 21:30:11,146 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:11,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610972_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520610972_0001_1, initTime=1463520611146
2016-05-17 21:30:11,147 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520610972_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:11,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610972_0001_1][Event:DAG_STARTED]: dagID=dag_1463520610972_0001_1, startTime=1463520611147
2016-05-17 21:30:11,147 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:11,147 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:11,147 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520610972_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:11,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:11,148 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DF3E4B1AA1B445D081049E911A288CB5 : {A94826EE69914242AAB3D3FCC74F42D9={InputName=A94826EE69914242AAB3D3FCC74F42D9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:11,148 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A94826EE69914242AAB3D3FCC74F42D9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:11,148 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5]
2016-05-17 21:30:11,148 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5]
2016-05-17 21:30:11,148 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5]
2016-05-17 21:30:11,148 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ..._RESULTS.0_VALUES_RESULTS, applicationId=application_1463520610972_0001, dagName=[A42D8DE156304B5D822423C2E093633E/CF6BD2B209EB4F54875173E3FF0B1422] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-17 21:30:11,148 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5]. Starting root input initializers: 1
2016-05-17 21:30:11,149 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:11,149 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A94826EE69914242AAB3D3FCC74F42D9 on vertex vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5]
2016-05-17 21:30:11,151 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:11,151 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A94826EE69914242AAB3D3FCC74F42D9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:11,151 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:11,152 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:11,153 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:11,153 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A94826EE69914242AAB3D3FCC74F42D9 on vertex vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5]
2016-05-17 21:30:11,154 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:11,154 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A94826EE69914242AAB3D3FCC74F42D9=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:11,154 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5] parallelism set to 1
2016-05-17 21:30:11,154 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:11,154 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610972_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520610972_0001_1_00, eventCount=1
2016-05-17 21:30:11,155 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5]
2016-05-17 21:30:11,155 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0655E5615DF843A2A7E86999978FDBC3, vertexId=vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:11,155 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0655E5615DF843A2A7E86999978FDBC3, vertexId=vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5]
2016-05-17 21:30:11,157 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0655E5615DF843A2A7E86999978FDBC3, vertexName=DF3E4B1AA1B445D081049E911A288CB5, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:11,157 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0655E5615DF843A2A7E86999978FDBC3, vertexId=vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5]
2016-05-17 21:30:11,158 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:11,158 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610972_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DF3E4B1AA1B445D081049E911A288CB5, vertexId=vertex_1463520610972_0001_1_00, initRequestedTime=1463520611148, initedTime=1463520611158, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:11,159 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5] with distanceFromRoot: 0
2016-05-17 21:30:11,159 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:11,159 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DF3E4B1AA1B445D081049E911A288CB5
2016-05-17 21:30:11,159 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5]
2016-05-17 21:30:11,160 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:11,160 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610972_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520610972_0001_1_00, startRequestedTime=1463520611159, startedTime=1463520611159
2016-05-17 21:30:11,160 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:11,161 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:11,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610972_0001_1][Event:TASK_STARTED]: vertexName=DF3E4B1AA1B445D081049E911A288CB5, taskId=task_1463520610972_0001_1_00_000000, scheduledTime=1463520611161, launchTime=1463520611161
2016-05-17 21:30:11,161 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520610972_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:11,161 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5]
2016-05-17 21:30:11,162 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A42D8DE156304B5D822423C2E093633E/CF6BD2B209EB4F54875173E3FF0B1422] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS, VertexName: DF3E4B1AA1B445D081049E911A288CB5, VertexParallelism: 1, TaskAttemptID:attempt_1463520610972_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A94826EE69914242AAB3D3FCC74F42D9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0655E5615DF843A2A7E86999978FDBC3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:11,162 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520610972_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:11,162 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:11,163 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:11,163 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:11,164 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:11,164 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520610972_0001_00_000001
2016-05-17 21:30:11,164 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610972_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:11,165 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520610972_0001_00_000001
2016-05-17 21:30:11,165 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:11,166 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520610972_0001, with dag id: dag_1463520610972_0001_1
2016-05-17 21:30:11,170 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610972_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:11,170 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:11,170 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610972_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520610972_0001_00_000001, launchTime=1463520611170
2016-05-17 21:30:11,171 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520610972_0001_00_000001
2016-05-17 21:30:11,172 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520610972_0001_1_00_000000_0] to container: [container_1463520610972_0001_00_000001]
2016-05-17 21:30:11,172 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610972_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:11,172 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520610972_0001_00_000001 given task: attempt_1463520610972_0001_1_00_000000_0
2016-05-17 21:30:11,173 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520610972_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520610972_0001_1_00_000000_0
2016-05-17 21:30:11,173 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520610972_0001_1_00_000000_0] started. Is using containerId: [container_1463520610972_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:11,173 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610972_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DF3E4B1AA1B445D081049E911A288CB5, taskAttemptId=attempt_1463520610972_0001_1_00_000000_0, startTime=1463520611172, containerId=container_1463520610972_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520610972_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:11 Starting to run new task attempt: attempt_1463520610972_0001_1_00_000000_0
2016-05-17 21:30:11,173 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520610972_0001_1_00_000000_0
2016-05-17 21:30:11,173 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520610972_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:11,173 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:11,173 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520610972_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:11,174 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:11,174 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:11,174 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:11,174 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:11,175 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:11,175 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:11,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A42D8DE156304B5D822423C2E093633E/CF6BD2B209EB4F54875173E3FF0B1422] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS, VertexName: DF3E4B1AA1B445D081049E911A288CB5, VertexParallelism: 1, TaskAttemptID:attempt_1463520610972_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A94826EE69914242AAB3D3FCC74F42D9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0655E5615DF843A2A7E86999978FDBC3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:11,176 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:11,176 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520610972_0001_1_00_000000_0
2016-05-17 21:30:11,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:11,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:11,178 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520610972_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:11,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:11,179 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A94826EE69914242AAB3D3FCC74F42D9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:11,179 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:11,179 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A94826EE69914242AAB3D3FCC74F42D9
2016-05-17 21:30:11,180 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A94826EE69914242AAB3D3FCC74F42D9
2016-05-17 21:30:11,179 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0655E5615DF843A2A7E86999978FDBC3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:11,181 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:11,181 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0655E5615DF843A2A7E86999978FDBC3
2016-05-17 21:30:11,181 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:11,181 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:11,181 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0655E5615DF843A2A7E86999978FDBC3
2016-05-17 21:30:11,184 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:11,186 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A94826EE69914242AAB3D3FCC74F42D9
2016-05-17 21:30:11,186 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A94826EE69914242AAB3D3FCC74F42D9
2016-05-17 21:30:11,193 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DF3E4B1AA1B445D081049E911A288CB5, ordinal: 0
2016-05-17 21:30:11,193 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DF3E4B1AA1B445D081049E911A288CB5, mem on start (mb), free: 161, total: 382, max: 672
2016-05-17 21:30:11,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:11,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:11,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:11,195 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:11,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0655E5615DF843A2A7E86999978FDBC3
2016-05-17 21:30:11,196 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:11,196 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:11,196 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:11,196 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:11,196 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:11,196 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:11,197 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:11,197 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:11,197 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A94826EE69914242AAB3D3FCC74F42D9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:11,197 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0655E5615DF843A2A7E86999978FDBC3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:11,197 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:11,197 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A94826EE69914242AAB3D3FCC74F42D9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:11,197 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A94826EE69914242AAB3D3FCC74F42D9
2016-05-17 21:30:11,197 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:11,197 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:11,198 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:11,198 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A94826EE69914242AAB3D3FCC74F42D9
2016-05-17 21:30:11,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:11,199 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520610972_0001_1_00_000000_0
2016-05-17 21:30:11,199 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:11,201 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1534fa6c
2016-05-17 21:30:11,201 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1534fa6c
2016-05-17 21:30:11,202 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:11,202 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:11,203 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:11,204 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: A94826EE69914242AAB3D3FCC74F42D9
2016-05-17 21:30:11,204 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS"], id: 0655E5615DF843A2A7E86999978FDBC3
2016-05-17 21:30:11,204 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS"]
2016-05-17 21:30:11,204 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:11,205 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:11,205 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DF3E4B1AA1B445D081049E911A288CB5, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:11,205 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:11,206 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:11,207 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:11,207 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:11,208 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:11,208 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520610972_0001_1_00_000000_0
2016-05-17 21:30:11,208 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520610972_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:11,209 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520610972_0001_1_00_000000_0
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:11,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463520611205, Process_Duration=4, Process_End_Time=1463520611209, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 21:30:11,211 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DF3E4B1AA1B445D081049E911A288CB5, mem on close (mb), free: 159, total: 382, max: 672
2016-05-17 21:30:11,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=DF3E4B1AA1B445D081049E911A288CB5, index=0
2016-05-17 21:30:11,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=DF3E4B1AA1B445D081049E911A288CB5, sourceVertex=A94826EE69914242AAB3D3FCC74F42D9
2016-05-17 21:30:11,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=DF3E4B1AA1B445D081049E911A288CB5, sourceVertex=0655E5615DF843A2A7E86999978FDBC3
2016-05-17 21:30:11,211 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520610972_0001_00_000001
2016-05-17 21:30:11,211 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520610972_0001_00_000001
2016-05-17 21:30:11,212 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520610972_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:30:11,213 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520610972_0001_00_000001 completed but with errors
2016-05-17 21:30:11,213 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:11,213 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610972_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DF3E4B1AA1B445D081049E911A288CB5, taskAttemptId=attempt_1463520610972_0001_1_00_000000_0, startTime=1463520611172, finishTime=1463520611213, timeTaken=41, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520611205, Process_Duration=4, Process_End_Time=1463520611209, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 21:30:11,213 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520610972_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:30:11,213 INFO container.AMContainerImpl (AMContainerImpl.java:transition(686)) - Container container_1463520610972_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:30:11,214 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610972_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520610972_0001_00_000001, stoppedTime=1463520611213, exitStatus=1
2016-05-17 21:30:11,214 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:11,214 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520610972_0001_00_000001 transitioned from RUNNING to COMPLETED via event C_COMPLETED
2016-05-17 21:30:11,214 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520610972_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:30:11,215 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:11,215 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610972_0001_1][Event:TASK_FINISHED]: vertexName=DF3E4B1AA1B445D081049E911A288CB5, taskId=task_1463520610972_0001_1_00_000000, startTime=1463520611172, finishTime=1463520611215, timeTaken=43, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
], counters=Counters: 0
2016-05-17 21:30:11,215 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520610972_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:30:11,215 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520610972_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:30:11,215 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5] : 1
2016-05-17 21:30:11,215 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5] because task failed: task_1463520610972_0001_1_00_000000
2016-05-17 21:30:11,216 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:30:11,216 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:30:11,216 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5]
2016-05-17 21:30:11,216 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=0655E5615DF843A2A7E86999978FDBC3, vertexId=vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5]
2016-05-17 21:30:11,217 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:11,218 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:11,218 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610972_0001_1][Event:VERTEX_FINISHED]: vertexName=DF3E4B1AA1B445D081049E911A288CB5, vertexId=vertex_1463520610972_0001_1_00, initRequestedTime=1463520611148, initedTime=1463520611158, startRequestedTime=1463520611159, startedTime=1463520611159, finishTime=1463520611217, timeTaken=58, status=FAILED, diagnostics=Task failed, taskId=task_1463520610972_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520611172, firstTasksToStart=[ task_1463520610972_0001_1_00_000000 ], lastTaskFinishTime=1463520611213, lastTasksToFinish=[ task_1463520610972_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:30:11,218 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:30:11,219 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 21:30:11,219 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:30:11,219 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520610972_0001_1
2016-05-17 21:30:11,219 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 0655E5615DF843A2A7E86999978FDBC3 for vertex: vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5]
2016-05-17 21:30:11,220 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520610972_0001_1, queueSize=0
2016-05-17 21:30:11,220 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:11,220 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520610972_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520610972_0001_1, startTime=1463520611147, finishTime=1463520611219, timeTaken=72, status=FAILED, diagnostics=Vertex failed, vertexName=DF3E4B1AA1B445D081049E911A288CB5, vertexId=vertex_1463520610972_0001_1_00, diagnostics=[Task failed, taskId=task_1463520610972_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520610972_0001_1_00 [DF3E4B1AA1B445D081049E911A288CB5] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 21:30:11,221 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520610972_0001_1 finished with state: FAILED
2016-05-17 21:30:11,221 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520610972_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:11,221 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520610972_0001_1_00_000000_0 failed: true container: container_1463520610972_0001_00_000001 numFailedTAs: 0
2016-05-17 21:30:11 Completed Dag: dag_1463520610972_0001_1
2016-05-17 21:30:11,221 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520610972_0001_1, dagState=FAILED
2016-05-17 21:30:11,222 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520610972_0001_1_post
2016-05-17 21:30:11,222 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:11,222 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:11,231 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ..._RESULTS.0_VALUES_RESULTS, applicationId=application_1463520610972_0001
2016-05-17 21:30:11,232 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:11,233 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:11,233 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:11,233 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:11,233 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:11,233 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:11,234 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:11,234 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:11,234 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:11,234 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:11,234 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:11,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:11,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:11,239 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A42D8DE156304B5D822423C2E093633E/CF6BD2B209EB4F54875173E3FF0B1422/.tez/application_1463520610972_0001
2016-05-17 21:30:11,239 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-17 21:30:11,241 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-17 21:30:11,241 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-17 21:30:11,242 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS/_temporary
2016-05-17 21:30:11,242 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.298
2016-05-17 21:30:11,242 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_RESULTS
2016-05-17 21:30:11,266 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:11,266 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:11,310 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:30:11,319 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 21:30:11,319 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:11,320 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:11,320 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:11,332 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:11,333 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL
2016-05-17 21:30:11,342 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:11,342 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:11,342 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL"]
2016-05-17 21:30:11,343 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:11,343 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:11,343 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:11,344 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-17 21:30:11,344 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:11,356 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:11,368 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:11,369 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:11,369 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:11,378 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FC84F2BF8B894F2B9750A95645AD064A/D71477DD13C14F6199D5EA659A0CC7C4/.tez/application_1463520611369_0001 doesn't exist and is created
2016-05-17 21:30:11,393 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:11,393 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:11,394 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FC84F2BF8B894F2B9750A95645AD064A/D71477DD13C14F6199D5EA659A0CC7C4/.tez/application_1463520611369_0001
2016-05-17 21:30:11,396 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520611369_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:11,406 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:11,407 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:11,408 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:11,408 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:11,409 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FC84F2BF8B894F2B9750A95645AD064A/D71477DD13C14F6199D5EA659A0CC7C4/.tez/application_1463520611369_0001/localmode-log-dir
2016-05-17 21:30:11,409 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FC84F2BF8B894F2B9750A95645AD064A/D71477DD13C14F6199D5EA659A0CC7C4/.tez/application_1463520611369_0001/localmode-log-dir/history.txt.appattempt_1463520611369_0001_000000, maxErrors=10
2016-05-17 21:30:11,409 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:11,410 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520611369_0001_000000, appSubmitTime=1463520611396, launchTime=1463520611396
2016-05-17 21:30:11,412 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:11,412 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:11,413 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:11,413 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:11,419 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:36061
2016-05-17 21:30:11,420 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:11,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520611369_0001_000000, startTime=1463520611430
2016-05-17 21:30:11,430 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:11,431 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:11,491 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:11,491 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:11,494 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:11,494 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:11,494 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_VALUES_ALL, applicationId=application_1463520611369_0001, dagName=[FC84F2BF8B894F2B9750A95645AD064A/D71477DD13C14F6199D5EA659A0CC7C4] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-17 21:30:11,510 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [FC84F2BF8B894F2B9750A95645AD064A/D71477DD13C14F6199D5EA659A0CC7C4] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-17 21:30:11,511 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520611369_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FC84F2BF8B894F2B9750A95645AD064A/D71477DD13C14F6199D5EA659A0CC7C4/.tez/application_1463520611369_0001/localmode-log-dir/dag_1463520611369_0001_1.dot
2016-05-17 21:30:11,512 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FC84F2BF8B894F2B9750A95645AD064A/D71477DD13C14F6199D5EA659A0CC7C4/.tez/application_1463520611369_0001/localmode-log-dir/dag_1463520611369_0001_1-tez-dag.pb.txt
2016-05-17 21:30:11,517 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520611369_0001_1
2016-05-17 21:30:11,517 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:11,517 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [FC84F2BF8B894F2B9750A95645AD064A/D71477DD13C14F6199D5EA659A0CC7C4] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-17 21:30:11 Running Dag: dag_1463520611369_0001_1
2016-05-17 21:30:11,529 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611369_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520611369_0001_1, submitTime=1463520611510
2016-05-17 21:30:11,529 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:11,529 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B9F29023016249F3A203209082F525DB
2016-05-17 21:30:11,529 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:11,529 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611369_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520611369_0001_1, initTime=1463520611529
2016-05-17 21:30:11,530 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520611369_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:11,530 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611369_0001_1][Event:DAG_STARTED]: dagID=dag_1463520611369_0001_1, startTime=1463520611530
2016-05-17 21:30:11,530 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:11,530 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520611369_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:11,529 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:11,531 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:11,530 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B9F29023016249F3A203209082F525DB : {9F1F198C8E604A32B078E8FB2B591971={InputName=9F1F198C8E604A32B078E8FB2B591971}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:11,531 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9F1F198C8E604A32B078E8FB2B591971, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:11,531 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB]
2016-05-17 21:30:11,531 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB]
2016-05-17 21:30:11,531 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB]
2016-05-17 21:30:11,531 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_VALUES_ALL, applicationId=application_1463520611369_0001, dagName=[FC84F2BF8B894F2B9750A95645AD064A/D71477DD13C14F6199D5EA659A0CC7C4] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-17 21:30:11,532 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB]. Starting root input initializers: 1
2016-05-17 21:30:11,532 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:11,532 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9F1F198C8E604A32B078E8FB2B591971 on vertex vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB]
2016-05-17 21:30:11,534 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:11,534 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9F1F198C8E604A32B078E8FB2B591971 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:11,535 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:11,535 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:11,536 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:11,537 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9F1F198C8E604A32B078E8FB2B591971 on vertex vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB]
2016-05-17 21:30:11,537 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:11,537 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9F1F198C8E604A32B078E8FB2B591971=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:11,537 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB] parallelism set to 1
2016-05-17 21:30:11,537 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:11,538 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611369_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520611369_0001_1_00, eventCount=1
2016-05-17 21:30:11,538 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB]
2016-05-17 21:30:11,538 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A91D092BE5E54280B73AAE904CE8E18C, vertexId=vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:11,538 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A91D092BE5E54280B73AAE904CE8E18C, vertexId=vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB]
2016-05-17 21:30:11,540 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A91D092BE5E54280B73AAE904CE8E18C, vertexName=B9F29023016249F3A203209082F525DB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:11,541 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A91D092BE5E54280B73AAE904CE8E18C, vertexId=vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB]
2016-05-17 21:30:11,542 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611369_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B9F29023016249F3A203209082F525DB, vertexId=vertex_1463520611369_0001_1_00, initRequestedTime=1463520611530, initedTime=1463520611542, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:11,542 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB] with distanceFromRoot: 0
2016-05-17 21:30:11,542 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:11,543 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:11,543 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B9F29023016249F3A203209082F525DB
2016-05-17 21:30:11,543 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB]
2016-05-17 21:30:11,544 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611369_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520611369_0001_1_00, startRequestedTime=1463520611543, startedTime=1463520611543
2016-05-17 21:30:11,544 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:11,544 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:11,545 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:11,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611369_0001_1][Event:TASK_STARTED]: vertexName=B9F29023016249F3A203209082F525DB, taskId=task_1463520611369_0001_1_00_000000, scheduledTime=1463520611545, launchTime=1463520611545
2016-05-17 21:30:11,545 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520611369_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:11,546 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB]
2016-05-17 21:30:11,546 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FC84F2BF8B894F2B9750A95645AD064A/D71477DD13C14F6199D5EA659A0CC7C4] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL, VertexName: B9F29023016249F3A203209082F525DB, VertexParallelism: 1, TaskAttemptID:attempt_1463520611369_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9F1F198C8E604A32B078E8FB2B591971, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A91D092BE5E54280B73AAE904CE8E18C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:11,546 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520611369_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:11,547 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:11,547 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:11,548 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520611369_0001, with dag id: dag_1463520611369_0001_1
2016-05-17 21:30:11,548 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:11,548 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:11,549 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520611369_0001_00_000001
2016-05-17 21:30:11,549 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520611369_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:11,551 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520611369_0001_00_000001
2016-05-17 21:30:11,551 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:11,552 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520611369_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:11,552 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:11,552 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611369_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520611369_0001_00_000001, launchTime=1463520611552
2016-05-17 21:30:11,553 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520611369_0001_00_000001
2016-05-17 21:30:11,553 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520611369_0001_1_00_000000_0] to container: [container_1463520611369_0001_00_000001]
2016-05-17 21:30:11,553 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520611369_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:11,553 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520611369_0001_00_000001 given task: attempt_1463520611369_0001_1_00_000000_0
2016-05-17 21:30:11,554 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520611369_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520611369_0001_1_00_000000_0
2016-05-17 21:30:11,554 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520611369_0001_1_00_000000_0] started. Is using containerId: [container_1463520611369_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:11,554 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611369_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B9F29023016249F3A203209082F525DB, taskAttemptId=attempt_1463520611369_0001_1_00_000000_0, startTime=1463520611554, containerId=container_1463520611369_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520611369_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:11,554 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:11 Starting to run new task attempt: attempt_1463520611369_0001_1_00_000000_0
2016-05-17 21:30:11,555 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520611369_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:11,555 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520611369_0001_1_00_000000_0
2016-05-17 21:30:11,555 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520611369_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:11,555 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:11,555 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:11,555 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:11,556 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:11,556 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:11,556 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:11,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FC84F2BF8B894F2B9750A95645AD064A/D71477DD13C14F6199D5EA659A0CC7C4] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL, VertexName: B9F29023016249F3A203209082F525DB, VertexParallelism: 1, TaskAttemptID:attempt_1463520611369_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9F1F198C8E604A32B078E8FB2B591971, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A91D092BE5E54280B73AAE904CE8E18C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:11,556 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:11,557 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520611369_0001_1_00_000000_0
2016-05-17 21:30:11,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:11,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:11,558 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520611369_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:11,558 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:11,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9F1F198C8E604A32B078E8FB2B591971, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:11,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:11,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9F1F198C8E604A32B078E8FB2B591971
2016-05-17 21:30:11,560 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9F1F198C8E604A32B078E8FB2B591971
2016-05-17 21:30:11,560 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:11,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A91D092BE5E54280B73AAE904CE8E18C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:11,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:11,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A91D092BE5E54280B73AAE904CE8E18C
2016-05-17 21:30:11,561 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:11,561 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A91D092BE5E54280B73AAE904CE8E18C
2016-05-17 21:30:11,567 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:11,568 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9F1F198C8E604A32B078E8FB2B591971
2016-05-17 21:30:11,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9F1F198C8E604A32B078E8FB2B591971
2016-05-17 21:30:11,570 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B9F29023016249F3A203209082F525DB, ordinal: 0
2016-05-17 21:30:11,570 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B9F29023016249F3A203209082F525DB, mem on start (mb), free: 300, total: 382, max: 672
2016-05-17 21:30:11,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:11,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:11,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:11,571 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:11,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A91D092BE5E54280B73AAE904CE8E18C
2016-05-17 21:30:11,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:11,571 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:11,571 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:11,571 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:11,571 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:11,571 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:11,572 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:11,572 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:11,572 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A91D092BE5E54280B73AAE904CE8E18C, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:11,572 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9F1F198C8E604A32B078E8FB2B591971, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:11,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:11,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9F1F198C8E604A32B078E8FB2B591971 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:11,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:11,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:11,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9F1F198C8E604A32B078E8FB2B591971
2016-05-17 21:30:11,573 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:11,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9F1F198C8E604A32B078E8FB2B591971
2016-05-17 21:30:11,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:11,573 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520611369_0001_1_00_000000_0
2016-05-17 21:30:11,575 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:11,576 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 9F1F198C8E604A32B078E8FB2B591971
2016-05-17 21:30:11,576 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL"], id: A91D092BE5E54280B73AAE904CE8E18C
2016-05-17 21:30:11,576 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL"]
2016-05-17 21:30:11,576 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:11,576 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:11,577 INFO input.MRInput (MRInput.java:checkAndAwaitRecordReaderInitialization(597)) - Awaiting RecordReader initialization
2016-05-17 21:30:11,577 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:11,577 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@77855502
2016-05-17 21:30:11,578 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@77855502
2016-05-17 21:30:11,578 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:11,579 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:11,579 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B9F29023016249F3A203209082F525DB, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:11,580 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:11,581 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:11,581 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:11,582 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:11,582 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:11,582 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520611369_0001_1_00_000000_0
2016-05-17 21:30:11,582 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520611369_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:11,583 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520611369_0001_1_00_000000_0
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 21:30:11,584 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463520611579, Process_Duration=4, Process_End_Time=1463520611583, Read_Duration=1, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 21:30:11,584 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B9F29023016249F3A203209082F525DB, mem on close (mb), free: 298, total: 382, max: 672
2016-05-17 21:30:11,584 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=B9F29023016249F3A203209082F525DB, index=0
2016-05-17 21:30:11,584 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=B9F29023016249F3A203209082F525DB, sourceVertex=9F1F198C8E604A32B078E8FB2B591971
2016-05-17 21:30:11,584 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=B9F29023016249F3A203209082F525DB, sourceVertex=A91D092BE5E54280B73AAE904CE8E18C
2016-05-17 21:30:11,585 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611369_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B9F29023016249F3A203209082F525DB, taskAttemptId=attempt_1463520611369_0001_1_00_000000_0, startTime=1463520611554, finishTime=1463520611584, timeTaken=30, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520611579, Process_Duration=4, Process_End_Time=1463520611583, Read_Duration=1, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 21:30:11,585 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520611369_0001_00_000001
2016-05-17 21:30:11,585 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520611369_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:30:11,585 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:11,585 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520611369_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:30:11,585 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611369_0001_1][Event:TASK_FINISHED]: vertexName=B9F29023016249F3A203209082F525DB, taskId=task_1463520611369_0001_1_00_000000, startTime=1463520611554, finishTime=1463520611585, timeTaken=31, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
], counters=Counters: 0
2016-05-17 21:30:11,585 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:11,586 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520611369_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:30:11,586 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB] : 1
2016-05-17 21:30:11,586 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:11,585 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520611369_0001_00_000001
2016-05-17 21:30:11,586 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB] because task failed: task_1463520611369_0001_1_00_000000
2016-05-17 21:30:11,587 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:30:11,587 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:30:11,587 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB]
2016-05-17 21:30:11,587 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=A91D092BE5E54280B73AAE904CE8E18C, vertexId=vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB]
2016-05-17 21:30:11,588 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611369_0001_1][Event:VERTEX_FINISHED]: vertexName=B9F29023016249F3A203209082F525DB, vertexId=vertex_1463520611369_0001_1_00, initRequestedTime=1463520611530, initedTime=1463520611542, startRequestedTime=1463520611543, startedTime=1463520611543, finishTime=1463520611588, timeTaken=45, status=FAILED, diagnostics=Task failed, taskId=task_1463520611369_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520611554, firstTasksToStart=[ task_1463520611369_0001_1_00_000000 ], lastTaskFinishTime=1463520611584, lastTasksToFinish=[ task_1463520611369_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:30:11,589 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:30:11,589 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520611369_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:30:11,589 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:11,590 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611369_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520611369_0001_00_000001, stoppedTime=1463520611590, exitStatus=0
2016-05-17 21:30:11,590 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520611369_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:11,590 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520611369_0001_1_00_000000_0 failed: true container: container_1463520611369_0001_00_000001 numFailedTAs: 0
2016-05-17 21:30:11,590 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520611369_0001_00_000001 completed but with errors
2016-05-17 21:30:11,590 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:11,591 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 21:30:11,591 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:30:11,591 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520611369_0001_1
2016-05-17 21:30:11,591 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: A91D092BE5E54280B73AAE904CE8E18C for vertex: vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB]
2016-05-17 21:30:11,592 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520611369_0001_1, queueSize=0
2016-05-17 21:30:11,592 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:11,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611369_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520611369_0001_1, startTime=1463520611530, finishTime=1463520611591, timeTaken=61, status=FAILED, diagnostics=Vertex failed, vertexName=B9F29023016249F3A203209082F525DB, vertexId=vertex_1463520611369_0001_1_00, diagnostics=[Task failed, taskId=task_1463520611369_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520611369_0001_1_00 [B9F29023016249F3A203209082F525DB] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 21:30:11,592 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520611369_0001_1 finished with state: FAILED
2016-05-17 21:30:11,592 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520611369_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:11,593 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520611369_0001_00_000001
2016-05-17 21:30:11,593 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520611369_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:30:11,593 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520611369_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:11 Completed Dag: dag_1463520611369_0001_1
2016-05-17 21:30:11,594 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520611369_0001_1, dagState=FAILED
2016-05-17 21:30:11,594 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520611369_0001_1_post
2016-05-17 21:30:11,594 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:11,594 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:11,594 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520611369_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:30:11,601 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...eld'_RESULTS.0_VALUES_ALL, applicationId=application_1463520611369_0001
2016-05-17 21:30:11,603 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:11,603 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:11,603 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:11,603 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:11,604 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:11,604 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:11,604 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:11,604 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:11,604 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:11,604 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:11,605 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:11,605 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:11,605 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:11,608 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FC84F2BF8B894F2B9750A95645AD064A/D71477DD13C14F6199D5EA659A0CC7C4/.tez/application_1463520611369_0001
2016-05-17 21:30:11,608 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-17 21:30:11,610 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-17 21:30:11,611 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-17 21:30:11,611 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL/_temporary
2016-05-17 21:30:11,611 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.269
2016-05-17 21:30:11,611 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_ALL
2016-05-17 21:30:11,635 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:11,635 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:11,667 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:30:11,668 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 21:30:11,668 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:11,669 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:11,669 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:11,681 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:11,681 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1
2016-05-17 21:30:11,691 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:11,691 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:11,691 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1"]
2016-05-17 21:30:11,691 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:11,691 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:11,692 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:11,692 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-17 21:30:11,692 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:11,703 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:11,707 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:11,708 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:11,708 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:11,717 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7F138E7866124988822BD905C52C5A5D/E7C116B743F84C9DA2E8F1C05EE84C5B/.tez/application_1463520611708_0001 doesn't exist and is created
2016-05-17 21:30:11,735 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:11,735 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:11,736 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7F138E7866124988822BD905C52C5A5D/E7C116B743F84C9DA2E8F1C05EE84C5B/.tez/application_1463520611708_0001
2016-05-17 21:30:11,738 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520611708_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:11,748 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:11,749 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:11,750 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:11,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:11,751 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7F138E7866124988822BD905C52C5A5D/E7C116B743F84C9DA2E8F1C05EE84C5B/.tez/application_1463520611708_0001/localmode-log-dir
2016-05-17 21:30:11,751 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7F138E7866124988822BD905C52C5A5D/E7C116B743F84C9DA2E8F1C05EE84C5B/.tez/application_1463520611708_0001/localmode-log-dir/history.txt.appattempt_1463520611708_0001_000000, maxErrors=10
2016-05-17 21:30:11,751 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:11,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520611708_0001_000000, appSubmitTime=1463520611738, launchTime=1463520611738
2016-05-17 21:30:11,755 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:11,756 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:11,757 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:11,757 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:11,758 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54058
2016-05-17 21:30:11,759 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:11,761 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520611708_0001_000000, startTime=1463520611761
2016-05-17 21:30:11,761 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:11,761 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:11,836 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:11,836 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:11,836 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...'field'_RESULTS.0_ARGS_-1, applicationId=application_1463520611708_0001, dagName=[7F138E7866124988822BD905C52C5A5D/E7C116B743F84C9DA2E8F1C05EE84C5B] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-17 21:30:11,850 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7F138E7866124988822BD905C52C5A5D/E7C116B743F84C9DA2E8F1C05EE84C5B] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-17 21:30:11,851 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520611708_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7F138E7866124988822BD905C52C5A5D/E7C116B743F84C9DA2E8F1C05EE84C5B/.tez/application_1463520611708_0001/localmode-log-dir/dag_1463520611708_0001_1.dot
2016-05-17 21:30:11,852 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7F138E7866124988822BD905C52C5A5D/E7C116B743F84C9DA2E8F1C05EE84C5B/.tez/application_1463520611708_0001/localmode-log-dir/dag_1463520611708_0001_1-tez-dag.pb.txt
2016-05-17 21:30:11,856 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520611708_0001_1
2016-05-17 21:30:11,856 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:11,856 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7F138E7866124988822BD905C52C5A5D/E7C116B743F84C9DA2E8F1C05EE84C5B] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-17 21:30:11 Running Dag: dag_1463520611708_0001_1
2016-05-17 21:30:11,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611708_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520611708_0001_1, submitTime=1463520611850
2016-05-17 21:30:11,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:11,868 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C8A67E80CFF7454B8C4791F562371E82
2016-05-17 21:30:11,868 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:11,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611708_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520611708_0001_1, initTime=1463520611868
2016-05-17 21:30:11,869 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:11,869 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520611708_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:11,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611708_0001_1][Event:DAG_STARTED]: dagID=dag_1463520611708_0001_1, startTime=1463520611869
2016-05-17 21:30:11,870 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:11,869 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:11,870 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520611708_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:11,870 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...'field'_RESULTS.0_ARGS_-1, applicationId=application_1463520611708_0001, dagName=[7F138E7866124988822BD905C52C5A5D/E7C116B743F84C9DA2E8F1C05EE84C5B] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-17 21:30:11,870 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C8A67E80CFF7454B8C4791F562371E82 : {411D6BC05DE04BF9B7CFFF4BBDD9D74E={InputName=411D6BC05DE04BF9B7CFFF4BBDD9D74E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:11,870 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 411D6BC05DE04BF9B7CFFF4BBDD9D74E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:11,870 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82]
2016-05-17 21:30:11,871 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82]
2016-05-17 21:30:11,871 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82]
2016-05-17 21:30:11,871 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82]. Starting root input initializers: 1
2016-05-17 21:30:11,871 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:11,872 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 411D6BC05DE04BF9B7CFFF4BBDD9D74E on vertex vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82]
2016-05-17 21:30:11,874 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:11,875 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 411D6BC05DE04BF9B7CFFF4BBDD9D74E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:11,875 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:11,876 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:11,877 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:11,877 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 411D6BC05DE04BF9B7CFFF4BBDD9D74E on vertex vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82]
2016-05-17 21:30:11,877 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:11,878 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {411D6BC05DE04BF9B7CFFF4BBDD9D74E=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:11,878 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82] parallelism set to 1
2016-05-17 21:30:11,878 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:11,878 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611708_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520611708_0001_1_00, eventCount=1
2016-05-17 21:30:11,878 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82]
2016-05-17 21:30:11,879 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0DD6E2237AFF43A8A83F53BF35FD77A1, vertexId=vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:11,879 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0DD6E2237AFF43A8A83F53BF35FD77A1, vertexId=vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82]
2016-05-17 21:30:11,881 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0DD6E2237AFF43A8A83F53BF35FD77A1, vertexName=C8A67E80CFF7454B8C4791F562371E82, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:11,881 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0DD6E2237AFF43A8A83F53BF35FD77A1, vertexId=vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82]
2016-05-17 21:30:11,882 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611708_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C8A67E80CFF7454B8C4791F562371E82, vertexId=vertex_1463520611708_0001_1_00, initRequestedTime=1463520611870, initedTime=1463520611882, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:11,882 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:11,883 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82] with distanceFromRoot: 0
2016-05-17 21:30:11,883 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:11,883 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C8A67E80CFF7454B8C4791F562371E82
2016-05-17 21:30:11,883 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82]
2016-05-17 21:30:11,884 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:11,884 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520611708_0001, with dag id: dag_1463520611708_0001_1
2016-05-17 21:30:11,884 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611708_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520611708_0001_1_00, startRequestedTime=1463520611883, startedTime=1463520611883
2016-05-17 21:30:11,884 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:11,885 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:11,885 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611708_0001_1][Event:TASK_STARTED]: vertexName=C8A67E80CFF7454B8C4791F562371E82, taskId=task_1463520611708_0001_1_00_000000, scheduledTime=1463520611885, launchTime=1463520611885
2016-05-17 21:30:11,885 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520611708_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:11,886 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82]
2016-05-17 21:30:11,886 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7F138E7866124988822BD905C52C5A5D/E7C116B743F84C9DA2E8F1C05EE84C5B] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1, VertexName: C8A67E80CFF7454B8C4791F562371E82, VertexParallelism: 1, TaskAttemptID:attempt_1463520611708_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=411D6BC05DE04BF9B7CFFF4BBDD9D74E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0DD6E2237AFF43A8A83F53BF35FD77A1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:11,886 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520611708_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:11,887 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:11,887 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:11,888 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:11,888 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:11,888 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520611708_0001_00_000001
2016-05-17 21:30:11,888 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520611708_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:11,890 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520611708_0001_00_000001
2016-05-17 21:30:11,890 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:11,891 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:11,891 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611708_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520611708_0001_00_000001, launchTime=1463520611890
2016-05-17 21:30:11,891 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520611708_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:11,891 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520611708_0001_00_000001
2016-05-17 21:30:11,892 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520611708_0001_1_00_000000_0] to container: [container_1463520611708_0001_00_000001]
2016-05-17 21:30:11,892 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520611708_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:11,892 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520611708_0001_00_000001 given task: attempt_1463520611708_0001_1_00_000000_0
2016-05-17 21:30:11,892 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520611708_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520611708_0001_1_00_000000_0
2016-05-17 21:30:11 Starting to run new task attempt: attempt_1463520611708_0001_1_00_000000_0
2016-05-17 21:30:11,893 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520611708_0001_1_00_000000_0] started. Is using containerId: [container_1463520611708_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:11,893 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520611708_0001_1_00_000000_0
2016-05-17 21:30:11,893 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:11,893 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:11,893 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:11,893 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:11,893 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:11,893 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:11,893 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:11,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7F138E7866124988822BD905C52C5A5D/E7C116B743F84C9DA2E8F1C05EE84C5B] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1, VertexName: C8A67E80CFF7454B8C4791F562371E82, VertexParallelism: 1, TaskAttemptID:attempt_1463520611708_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=411D6BC05DE04BF9B7CFFF4BBDD9D74E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0DD6E2237AFF43A8A83F53BF35FD77A1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:11,894 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:11,894 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611708_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C8A67E80CFF7454B8C4791F562371E82, taskAttemptId=attempt_1463520611708_0001_1_00_000000_0, startTime=1463520611892, containerId=container_1463520611708_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520611708_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:11,894 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520611708_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:11,894 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520611708_0001_1_00_000000_0
2016-05-17 21:30:11,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:11,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:11,895 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520611708_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:11,895 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520611708_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:11,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:11,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0DD6E2237AFF43A8A83F53BF35FD77A1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:11,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:11,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0DD6E2237AFF43A8A83F53BF35FD77A1
2016-05-17 21:30:11,896 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:11,896 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0DD6E2237AFF43A8A83F53BF35FD77A1
2016-05-17 21:30:11,898 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:11,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=411D6BC05DE04BF9B7CFFF4BBDD9D74E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:11,902 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:11,902 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 411D6BC05DE04BF9B7CFFF4BBDD9D74E
2016-05-17 21:30:11,902 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 411D6BC05DE04BF9B7CFFF4BBDD9D74E
2016-05-17 21:30:11,905 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:11,905 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 411D6BC05DE04BF9B7CFFF4BBDD9D74E
2016-05-17 21:30:11,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 411D6BC05DE04BF9B7CFFF4BBDD9D74E
2016-05-17 21:30:11,907 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:11,907 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0DD6E2237AFF43A8A83F53BF35FD77A1
2016-05-17 21:30:11,909 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C8A67E80CFF7454B8C4791F562371E82, ordinal: 0
2016-05-17 21:30:11,909 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8A67E80CFF7454B8C4791F562371E82, mem on start (mb), free: 240, total: 382, max: 672
2016-05-17 21:30:11,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:11,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:11,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:11,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:11,910 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:11,910 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:11,910 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:11,910 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:11,910 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:11,911 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:11,911 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:11,911 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0DD6E2237AFF43A8A83F53BF35FD77A1, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:11,911 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 411D6BC05DE04BF9B7CFFF4BBDD9D74E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:11,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:11,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 411D6BC05DE04BF9B7CFFF4BBDD9D74E
2016-05-17 21:30:11,911 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:11,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 411D6BC05DE04BF9B7CFFF4BBDD9D74E
2016-05-17 21:30:11,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 411D6BC05DE04BF9B7CFFF4BBDD9D74E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:11,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:11,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:11,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:11,913 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520611708_0001_1_00_000000_0
2016-05-17 21:30:11,913 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:11,914 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@738694c5
2016-05-17 21:30:11,915 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@738694c5
2016-05-17 21:30:11,915 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:11,916 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:11,918 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:11,918 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 411D6BC05DE04BF9B7CFFF4BBDD9D74E
2016-05-17 21:30:11,919 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1"], id: 0DD6E2237AFF43A8A83F53BF35FD77A1
2016-05-17 21:30:11,919 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1"]
2016-05-17 21:30:11,920 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:11,920 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:11,920 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C8A67E80CFF7454B8C4791F562371E82, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:11,923 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:11,924 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:11,925 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520611708_0001_1_00_000000_0
2016-05-17 21:30:11,925 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520611708_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:11,926 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520611708_0001_1_00_000000_0
2016-05-17 21:30:11,926 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8A67E80CFF7454B8C4791F562371E82, mem on close (mb), free: 239, total: 382, max: 672
2016-05-17 21:30:11 Completed running task attempt: attempt_1463520611708_0001_1_00_000000_0
2016-05-17 21:30:11,927 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520611708_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:11,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520611920, Process_Duration=6, Process_End_Time=1463520611926, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:11,928 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611708_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C8A67E80CFF7454B8C4791F562371E82, taskAttemptId=attempt_1463520611708_0001_1_00_000000_0, startTime=1463520611892, finishTime=1463520611928, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520611920, Process_Duration=6, Process_End_Time=1463520611926, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:11,928 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:11,928 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:11,928 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520611708_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:11,929 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520611708_0001_1_00_000000_0
2016-05-17 21:30:11,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611708_0001_1][Event:TASK_FINISHED]: vertexName=C8A67E80CFF7454B8C4791F562371E82, taskId=task_1463520611708_0001_1_00_000000, startTime=1463520611892, finishTime=1463520611929, timeTaken=37, status=SUCCEEDED, successfulAttemptID=attempt_1463520611708_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520611920, Process_Duration=6, Process_End_Time=1463520611926, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:11,930 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520611708_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:11,930 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82] : 1
2016-05-17 21:30:11,930 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82]
2016-05-17 21:30:11,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611708_0001_1][Event:VERTEX_FINISHED]: vertexName=C8A67E80CFF7454B8C4791F562371E82, vertexId=vertex_1463520611708_0001_1_00, initRequestedTime=1463520611870, initedTime=1463520611882, startRequestedTime=1463520611883, startedTime=1463520611883, finishTime=1463520611930, timeTaken=47, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520611920, Process_Duration=6, Process_End_Time=1463520611926, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520611892, firstTasksToStart=[ task_1463520611708_0001_1_00_000000 ], lastTaskFinishTime=1463520611928, lastTasksToFinish=[ task_1463520611708_0001_1_00_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520611708_0001_1_00_000000 ], longestDurationTasks=[ task_1463520611708_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:11,931 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:11,931 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:11,931 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520611708_0001_1
2016-05-17 21:30:11,931 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611708_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520611708_0001_1
2016-05-17 21:30:11,931 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0DD6E2237AFF43A8A83F53BF35FD77A1 for vertex: vertex_1463520611708_0001_1_00 [C8A67E80CFF7454B8C4791F562371E82]
2016-05-17 21:30:11,932 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:11,928 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:11,931 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:11,933 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520611708_0001_00_000001
2016-05-17 21:30:11,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:11,941 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520611708_0001_1, queueSize=0
2016-05-17 21:30:11,941 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:11,941 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611708_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520611708_0001_1, startTime=1463520611869, finishTime=1463520611939, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520611920, Process_Duration=6, Process_End_Time=1463520611926, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:11,941 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520611708_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:11,942 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520611708_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:11,942 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520611708_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:11,942 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520611708_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520611708_0001_00_000001, stoppedTime=1463520611942, exitStatus=0
2016-05-17 21:30:11,942 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520611708_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:11 Completed Dag: dag_1463520611708_0001_1
2016-05-17 21:30:11,942 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520611708_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:11,943 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520611708_0001_1_post
2016-05-17 21:30:11,943 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:11,943 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:11,942 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:11,943 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520611708_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:30:11,944 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520611708_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:11,944 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520611708_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:11,944 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520611708_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:30:11,944 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520611708_0001_00_000001, Exiting
2016-05-17 21:30:11,944 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520611708_0001_00_000001
2016-05-17 21:30:11,945 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520611708_0001_00_000001
2016-05-17 21:30:11,946 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520611708_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:11,947 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520611708_0001_00_000001 completed successfully
2016-05-17 21:30:11,947 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520611708_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:30:11,950 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...'field'_RESULTS.0_ARGS_-1, applicationId=application_1463520611708_0001
2016-05-17 21:30:11,951 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:11,951 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:11,951 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:11,951 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:11,951 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:11,952 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:11,952 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:11,952 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:11,952 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:11,952 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:11,952 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:11,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:11,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:11,956 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7F138E7866124988822BD905C52C5A5D/E7C116B743F84C9DA2E8F1C05EE84C5B/.tez/application_1463520611708_0001
2016-05-17 21:30:11,956 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1/_temporary
2016-05-17 21:30:11,957 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.265
2016-05-17 21:30:11,980 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:11,981 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:12,020 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_-1
2016-05-17 21:30:12,040 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:12,040 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:12,067 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-17 21:30:12,067 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-17 21:30:12,068 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:12,068 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:12,068 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_'field2'
2016-05-17 21:30:12,092 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:12,092 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:12,122 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 21:30:12,125 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 21:30:12,125 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:12,125 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:12,125 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:12,138 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:12,138 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS
2016-05-17 21:30:12,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:12,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:12,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS"]
2016-05-17 21:30:12,148 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:12,148 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:12,148 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:12,148 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-17 21:30:12,148 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:12,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:12,165 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:12,166 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:12,166 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:12,176 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/F7A747CFA27D4E5D838E67FCF8117D23/C1D61ABC74EF4C67B7C91EDB93BD8DEE/.tez/application_1463520612165_0001 doesn't exist and is created
2016-05-17 21:30:12,193 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:12,194 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:12,194 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/F7A747CFA27D4E5D838E67FCF8117D23/C1D61ABC74EF4C67B7C91EDB93BD8DEE/.tez/application_1463520612165_0001
2016-05-17 21:30:12,198 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520612165_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:12,212 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:12,213 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:12,215 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:12,215 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:12,215 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/F7A747CFA27D4E5D838E67FCF8117D23/C1D61ABC74EF4C67B7C91EDB93BD8DEE/.tez/application_1463520612165_0001/localmode-log-dir
2016-05-17 21:30:12,216 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/F7A747CFA27D4E5D838E67FCF8117D23/C1D61ABC74EF4C67B7C91EDB93BD8DEE/.tez/application_1463520612165_0001/localmode-log-dir/history.txt.appattempt_1463520612165_0001_000000, maxErrors=10
2016-05-17 21:30:12,216 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:12,216 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520612165_0001_000000, appSubmitTime=1463520612197, launchTime=1463520612197
2016-05-17 21:30:12,220 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:12,222 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:12,224 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:12,225 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:12,231 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56731
2016-05-17 21:30:12,233 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:12,235 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520612165_0001_000000, startTime=1463520612235
2016-05-17 21:30:12,235 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:12,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:12,294 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:12,294 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:12,294 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_ARGS_RESULTS, applicationId=application_1463520612165_0001, dagName=[F7A747CFA27D4E5D838E67FCF8117D23/C1D61ABC74EF4C67B7C91EDB93BD8DEE] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-17 21:30:12,315 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [F7A747CFA27D4E5D838E67FCF8117D23/C1D61ABC74EF4C67B7C91EDB93BD8DEE] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-17 21:30:12,316 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520612165_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/F7A747CFA27D4E5D838E67FCF8117D23/C1D61ABC74EF4C67B7C91EDB93BD8DEE/.tez/application_1463520612165_0001/localmode-log-dir/dag_1463520612165_0001_1.dot
2016-05-17 21:30:12,316 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/F7A747CFA27D4E5D838E67FCF8117D23/C1D61ABC74EF4C67B7C91EDB93BD8DEE/.tez/application_1463520612165_0001/localmode-log-dir/dag_1463520612165_0001_1-tez-dag.pb.txt
2016-05-17 21:30:12,323 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520612165_0001_1
2016-05-17 21:30:12,323 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:12,323 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [F7A747CFA27D4E5D838E67FCF8117D23/C1D61ABC74EF4C67B7C91EDB93BD8DEE] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-17 21:30:12 Running Dag: dag_1463520612165_0001_1
2016-05-17 21:30:12,335 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612165_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520612165_0001_1, submitTime=1463520612315
2016-05-17 21:30:12,335 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:12,336 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A29951EE3CE941B5916238B6578A8D7B
2016-05-17 21:30:12,337 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:12,338 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612165_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520612165_0001_1, initTime=1463520612336
2016-05-17 21:30:12,338 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:12,338 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520612165_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:12,339 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612165_0001_1][Event:DAG_STARTED]: dagID=dag_1463520612165_0001_1, startTime=1463520612339
2016-05-17 21:30:12,339 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:12,339 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520612165_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:12,339 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:12,339 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A29951EE3CE941B5916238B6578A8D7B : {2B7E63757EAC490FBA06DA8BAC3673DF={InputName=2B7E63757EAC490FBA06DA8BAC3673DF}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:12,340 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2B7E63757EAC490FBA06DA8BAC3673DF, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:12,340 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B]
2016-05-17 21:30:12,340 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B]
2016-05-17 21:30:12,339 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_ARGS_RESULTS, applicationId=application_1463520612165_0001, dagName=[F7A747CFA27D4E5D838E67FCF8117D23/C1D61ABC74EF4C67B7C91EDB93BD8DEE] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-17 21:30:12,340 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B]
2016-05-17 21:30:12,340 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B]. Starting root input initializers: 1
2016-05-17 21:30:12,342 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:12,342 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2B7E63757EAC490FBA06DA8BAC3673DF on vertex vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B]
2016-05-17 21:30:12,345 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:12,346 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2B7E63757EAC490FBA06DA8BAC3673DF asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:12,346 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:12,346 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:12,348 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:12,348 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2B7E63757EAC490FBA06DA8BAC3673DF on vertex vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B]
2016-05-17 21:30:12,349 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:12,349 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2B7E63757EAC490FBA06DA8BAC3673DF=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:12,349 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B] parallelism set to 1
2016-05-17 21:30:12,349 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:12,349 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612165_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520612165_0001_1_00, eventCount=1
2016-05-17 21:30:12,349 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B]
2016-05-17 21:30:12,349 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=813A5F18208744D3806A40EFDDA161E6, vertexId=vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:12,349 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=813A5F18208744D3806A40EFDDA161E6, vertexId=vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B]
2016-05-17 21:30:12,351 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=813A5F18208744D3806A40EFDDA161E6, vertexName=A29951EE3CE941B5916238B6578A8D7B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:12,351 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=813A5F18208744D3806A40EFDDA161E6, vertexId=vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B]
2016-05-17 21:30:12,352 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612165_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A29951EE3CE941B5916238B6578A8D7B, vertexId=vertex_1463520612165_0001_1_00, initRequestedTime=1463520612339, initedTime=1463520612352, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:12,352 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B] with distanceFromRoot: 0
2016-05-17 21:30:12,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:12,352 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:12,353 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A29951EE3CE941B5916238B6578A8D7B
2016-05-17 21:30:12,353 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B]
2016-05-17 21:30:12,353 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612165_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520612165_0001_1_00, startRequestedTime=1463520612353, startedTime=1463520612353
2016-05-17 21:30:12,353 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:12,353 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:12,354 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:12,354 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612165_0001_1][Event:TASK_STARTED]: vertexName=A29951EE3CE941B5916238B6578A8D7B, taskId=task_1463520612165_0001_1_00_000000, scheduledTime=1463520612354, launchTime=1463520612354
2016-05-17 21:30:12,356 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520612165_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:12,356 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B]
2016-05-17 21:30:12,357 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [F7A747CFA27D4E5D838E67FCF8117D23/C1D61ABC74EF4C67B7C91EDB93BD8DEE] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS, VertexName: A29951EE3CE941B5916238B6578A8D7B, VertexParallelism: 1, TaskAttemptID:attempt_1463520612165_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2B7E63757EAC490FBA06DA8BAC3673DF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=813A5F18208744D3806A40EFDDA161E6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:12,358 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520612165_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:12,358 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520612165_0001, with dag id: dag_1463520612165_0001_1
2016-05-17 21:30:12,362 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:12,363 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:12,363 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:12,364 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:12,364 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520612165_0001_00_000001
2016-05-17 21:30:12,366 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520612165_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:12,369 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520612165_0001_00_000001
2016-05-17 21:30:12,369 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:12,370 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520612165_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:12,370 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:12,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612165_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520612165_0001_00_000001, launchTime=1463520612370
2016-05-17 21:30:12,372 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520612165_0001_00_000001
2016-05-17 21:30:12,372 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520612165_0001_1_00_000000_0] to container: [container_1463520612165_0001_00_000001]
2016-05-17 21:30:12,372 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520612165_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:12,372 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520612165_0001_00_000001 given task: attempt_1463520612165_0001_1_00_000000_0
2016-05-17 21:30:12,372 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520612165_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520612165_0001_1_00_000000_0
2016-05-17 21:30:12 Starting to run new task attempt: attempt_1463520612165_0001_1_00_000000_0
2016-05-17 21:30:12,373 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520612165_0001_1_00_000000_0
2016-05-17 21:30:12,373 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:12,374 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:12,374 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:12,374 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:12,374 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:12,375 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:12,375 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F7A747CFA27D4E5D838E67FCF8117D23/C1D61ABC74EF4C67B7C91EDB93BD8DEE] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS, VertexName: A29951EE3CE941B5916238B6578A8D7B, VertexParallelism: 1, TaskAttemptID:attempt_1463520612165_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2B7E63757EAC490FBA06DA8BAC3673DF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=813A5F18208744D3806A40EFDDA161E6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:12,375 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:12,376 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520612165_0001_1_00_000000_0
2016-05-17 21:30:12,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:12,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:12,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:12,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2B7E63757EAC490FBA06DA8BAC3673DF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:12,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:12,373 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520612165_0001_1_00_000000_0] started. Is using containerId: [container_1463520612165_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:12,381 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612165_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A29951EE3CE941B5916238B6578A8D7B, taskAttemptId=attempt_1463520612165_0001_1_00_000000_0, startTime=1463520612372, containerId=container_1463520612165_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520612165_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:12,382 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:12,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2B7E63757EAC490FBA06DA8BAC3673DF
2016-05-17 21:30:12,382 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2B7E63757EAC490FBA06DA8BAC3673DF
2016-05-17 21:30:12,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=813A5F18208744D3806A40EFDDA161E6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:12,382 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520612165_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:12,383 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520612165_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:12,381 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:12,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:12,384 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520612165_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:12,384 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:12,385 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2B7E63757EAC490FBA06DA8BAC3673DF
2016-05-17 21:30:12,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2B7E63757EAC490FBA06DA8BAC3673DF
2016-05-17 21:30:12,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 813A5F18208744D3806A40EFDDA161E6
2016-05-17 21:30:12,385 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:12,385 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 813A5F18208744D3806A40EFDDA161E6
2016-05-17 21:30:12,392 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A29951EE3CE941B5916238B6578A8D7B, ordinal: 0
2016-05-17 21:30:12,395 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A29951EE3CE941B5916238B6578A8D7B, mem on start (mb), free: 153, total: 382, max: 672
2016-05-17 21:30:12,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:12,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:12,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:12,398 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:12,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 813A5F18208744D3806A40EFDDA161E6
2016-05-17 21:30:12,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:12,399 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:12,399 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:12,399 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:12,399 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:12,399 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:12,399 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:12,399 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:12,399 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2B7E63757EAC490FBA06DA8BAC3673DF, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:12,400 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 813A5F18208744D3806A40EFDDA161E6, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:12,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:12,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2B7E63757EAC490FBA06DA8BAC3673DF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:12,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2B7E63757EAC490FBA06DA8BAC3673DF
2016-05-17 21:30:12,400 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:12,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2B7E63757EAC490FBA06DA8BAC3673DF
2016-05-17 21:30:12,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:12,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:12,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:12,401 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520612165_0001_1_00_000000_0
2016-05-17 21:30:12,401 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:12,401 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1031b4db
2016-05-17 21:30:12,402 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1031b4db
2016-05-17 21:30:12,402 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:12,402 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:12,404 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:12,404 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2B7E63757EAC490FBA06DA8BAC3673DF
2016-05-17 21:30:12,404 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS"], id: 813A5F18208744D3806A40EFDDA161E6
2016-05-17 21:30:12,405 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS"]
2016-05-17 21:30:12,405 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:12,405 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:12,405 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A29951EE3CE941B5916238B6578A8D7B, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:12,408 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:12,409 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:12,409 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520612165_0001_1_00_000000_0
2016-05-17 21:30:12,409 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520612165_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:12,410 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520612165_0001_1_00_000000_0
2016-05-17 21:30:12,410 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A29951EE3CE941B5916238B6578A8D7B, mem on close (mb), free: 151, total: 382, max: 672
2016-05-17 21:30:12 Completed running task attempt: attempt_1463520612165_0001_1_00_000000_0
2016-05-17 21:30:12,411 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520612165_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:12,412 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520612405, Process_Duration=5, Process_End_Time=1463520612410, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:12,413 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612165_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A29951EE3CE941B5916238B6578A8D7B, taskAttemptId=attempt_1463520612165_0001_1_00_000000_0, startTime=1463520612372, finishTime=1463520612412, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520612405, Process_Duration=5, Process_End_Time=1463520612410, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:12,413 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:12,413 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:12,413 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:12,413 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520612165_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:12,414 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520612165_0001_00_000001
2016-05-17 21:30:12,414 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:12,414 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520612165_0001_1_00_000000_0
2016-05-17 21:30:12,415 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:12,415 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612165_0001_1][Event:TASK_FINISHED]: vertexName=A29951EE3CE941B5916238B6578A8D7B, taskId=task_1463520612165_0001_1_00_000000, startTime=1463520612372, finishTime=1463520612414, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463520612165_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520612405, Process_Duration=5, Process_End_Time=1463520612410, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:12,415 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520612165_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:12,415 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B] : 1
2016-05-17 21:30:12,415 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B]
2016-05-17 21:30:12,416 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:12,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612165_0001_1][Event:VERTEX_FINISHED]: vertexName=A29951EE3CE941B5916238B6578A8D7B, vertexId=vertex_1463520612165_0001_1_00, initRequestedTime=1463520612339, initedTime=1463520612352, startRequestedTime=1463520612353, startedTime=1463520612353, finishTime=1463520612415, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520612405, Process_Duration=5, Process_End_Time=1463520612410, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520612372, firstTasksToStart=[ task_1463520612165_0001_1_00_000000 ], lastTaskFinishTime=1463520612412, lastTasksToFinish=[ task_1463520612165_0001_1_00_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520612165_0001_1_00_000000 ], longestDurationTasks=[ task_1463520612165_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:12,416 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:12,416 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520612165_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:12,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612165_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520612165_0001_00_000001, stoppedTime=1463520612416, exitStatus=0
2016-05-17 21:30:12,416 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:12,417 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520612165_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:12,417 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:12,417 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520612165_0001_1
2016-05-17 21:30:12,417 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612165_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520612165_0001_1
2016-05-17 21:30:12,417 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 813A5F18208744D3806A40EFDDA161E6 for vertex: vertex_1463520612165_0001_1_00 [A29951EE3CE941B5916238B6578A8D7B]
2016-05-17 21:30:12,424 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520612165_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:12,424 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520612165_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:12,424 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520612165_0001_00_000001, Exiting
2016-05-17 21:30:12,425 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520612165_0001_00_000001
2016-05-17 21:30:12,425 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520612165_0001_00_000001
2016-05-17 21:30:12,425 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520612165_0001_1, queueSize=0
2016-05-17 21:30:12,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:12,425 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612165_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520612165_0001_1, startTime=1463520612339, finishTime=1463520612423, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=672, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520612405, Process_Duration=5, Process_End_Time=1463520612410, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:12,425 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520612165_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:12,426 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520612165_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:12,426 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520612165_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:30:12 Completed Dag: dag_1463520612165_0001_1
2016-05-17 21:30:12,426 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520612165_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:12,426 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520612165_0001_1_post
2016-05-17 21:30:12,426 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:12,426 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:12,426 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520612165_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:30:12,427 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520612165_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:12,427 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520612165_0001_00_000001 completed successfully
2016-05-17 21:30:12,427 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520612165_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:30:12,432 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.0_ARGS_RESULTS, applicationId=application_1463520612165_0001
2016-05-17 21:30:12,433 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:12,434 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:12,434 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:12,434 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:12,434 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:12,434 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:12,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:12,435 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:12,435 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:12,435 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:12,435 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:12,435 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:12,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:12,439 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/F7A747CFA27D4E5D838E67FCF8117D23/C1D61ABC74EF4C67B7C91EDB93BD8DEE/.tez/application_1463520612165_0001
2016-05-17 21:30:12,439 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS/_temporary
2016-05-17 21:30:12,440 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.292
2016-05-17 21:30:12,502 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_RESULTS
2016-05-17 21:30:12,517 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:12,517 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:12,526 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:12,527 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:12,555 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 21:30:12,558 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-05-17 21:30:12,558 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:12,558 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:12,558 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_ALL
2016-05-17 21:30:12,592 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:12,592 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:12,619 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 21:30:12,628 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 21:30:12,629 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:12,629 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:12,629 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:12,643 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:12,643 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1
2016-05-17 21:30:12,652 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:12,652 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:12,652 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1"]
2016-05-17 21:30:12,653 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:12,653 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:12,653 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:12,656 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-17 21:30:12,656 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:12,666 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:12,672 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:12,672 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:12,672 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:12,682 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DE5BC638820E47F19404861799CBC5B7/C8FD204DDDBE48609D4E9B30EC4ECD9C/.tez/application_1463520612672_0001 doesn't exist and is created
2016-05-17 21:30:12,697 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:12,697 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:12,697 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DE5BC638820E47F19404861799CBC5B7/C8FD204DDDBE48609D4E9B30EC4ECD9C/.tez/application_1463520612672_0001
2016-05-17 21:30:12,700 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520612672_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:12,710 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:12,711 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:12,713 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:12,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:12,713 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DE5BC638820E47F19404861799CBC5B7/C8FD204DDDBE48609D4E9B30EC4ECD9C/.tez/application_1463520612672_0001/localmode-log-dir
2016-05-17 21:30:12,713 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DE5BC638820E47F19404861799CBC5B7/C8FD204DDDBE48609D4E9B30EC4ECD9C/.tez/application_1463520612672_0001/localmode-log-dir/history.txt.appattempt_1463520612672_0001_000000, maxErrors=10
2016-05-17 21:30:12,713 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:12,714 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520612672_0001_000000, appSubmitTime=1463520612700, launchTime=1463520612700
2016-05-17 21:30:12,718 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54127
2016-05-17 21:30:12,724 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:12,726 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:12,726 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:12,727 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:12,727 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:12,729 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520612672_0001_000000, startTime=1463520612729
2016-05-17 21:30:12,729 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:12,729 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:12,797 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:12,797 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:12,797 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...'_RESULTS.ALL_'field2'_-1, applicationId=application_1463520612672_0001, dagName=[DE5BC638820E47F19404861799CBC5B7/C8FD204DDDBE48609D4E9B30EC4ECD9C] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-17 21:30:12,811 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [DE5BC638820E47F19404861799CBC5B7/C8FD204DDDBE48609D4E9B30EC4ECD9C] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-17 21:30:12,812 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520612672_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DE5BC638820E47F19404861799CBC5B7/C8FD204DDDBE48609D4E9B30EC4ECD9C/.tez/application_1463520612672_0001/localmode-log-dir/dag_1463520612672_0001_1.dot
2016-05-17 21:30:12,813 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DE5BC638820E47F19404861799CBC5B7/C8FD204DDDBE48609D4E9B30EC4ECD9C/.tez/application_1463520612672_0001/localmode-log-dir/dag_1463520612672_0001_1-tez-dag.pb.txt
2016-05-17 21:30:12,817 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520612672_0001_1
2016-05-17 21:30:12,817 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:12,817 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [DE5BC638820E47F19404861799CBC5B7/C8FD204DDDBE48609D4E9B30EC4ECD9C] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-17 21:30:12 Running Dag: dag_1463520612672_0001_1
2016-05-17 21:30:12,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612672_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520612672_0001_1, submitTime=1463520612811
2016-05-17 21:30:12,827 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:12,828 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B5D2E9C4C0BE4115AA4980D0E3C4CF98
2016-05-17 21:30:12,828 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:12,828 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612672_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520612672_0001_1, initTime=1463520612827
2016-05-17 21:30:12,828 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:12,828 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520612672_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:12,829 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612672_0001_1][Event:DAG_STARTED]: dagID=dag_1463520612672_0001_1, startTime=1463520612829
2016-05-17 21:30:12,829 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:12,829 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520612672_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:12,830 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B5D2E9C4C0BE4115AA4980D0E3C4CF98 : {2278C2920212466F8BFEF5C443937C8F={InputName=2278C2920212466F8BFEF5C443937C8F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:12,830 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...'_RESULTS.ALL_'field2'_-1, applicationId=application_1463520612672_0001, dagName=[DE5BC638820E47F19404861799CBC5B7/C8FD204DDDBE48609D4E9B30EC4ECD9C] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-17 21:30:12,829 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:12,830 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2278C2920212466F8BFEF5C443937C8F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:12,831 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98]
2016-05-17 21:30:12,831 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98]
2016-05-17 21:30:12,831 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98]
2016-05-17 21:30:12,832 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98]. Starting root input initializers: 1
2016-05-17 21:30:12,832 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:12,832 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2278C2920212466F8BFEF5C443937C8F on vertex vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98]
2016-05-17 21:30:12,834 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:12,834 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2278C2920212466F8BFEF5C443937C8F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:12,835 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:12,835 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:12,836 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:12,836 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2278C2920212466F8BFEF5C443937C8F on vertex vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98]
2016-05-17 21:30:12,837 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:12,837 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2278C2920212466F8BFEF5C443937C8F=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:12,837 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98] parallelism set to 1
2016-05-17 21:30:12,837 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:12,837 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612672_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520612672_0001_1_00, eventCount=1
2016-05-17 21:30:12,837 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98]
2016-05-17 21:30:12,838 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=EA136162D58D4D49BDA3DA1AAA75FF6B, vertexId=vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:12,838 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=EA136162D58D4D49BDA3DA1AAA75FF6B, vertexId=vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98]
2016-05-17 21:30:12,841 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=EA136162D58D4D49BDA3DA1AAA75FF6B, vertexName=B5D2E9C4C0BE4115AA4980D0E3C4CF98, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:12,841 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=EA136162D58D4D49BDA3DA1AAA75FF6B, vertexId=vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98]
2016-05-17 21:30:12,842 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612672_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B5D2E9C4C0BE4115AA4980D0E3C4CF98, vertexId=vertex_1463520612672_0001_1_00, initRequestedTime=1463520612830, initedTime=1463520612842, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:12,842 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98] with distanceFromRoot: 0
2016-05-17 21:30:12,842 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:12,842 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:12,842 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B5D2E9C4C0BE4115AA4980D0E3C4CF98
2016-05-17 21:30:12,842 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98]
2016-05-17 21:30:12,842 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:12,843 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612672_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520612672_0001_1_00, startRequestedTime=1463520612842, startedTime=1463520612842
2016-05-17 21:30:12,843 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:12,843 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:12,843 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612672_0001_1][Event:TASK_STARTED]: vertexName=B5D2E9C4C0BE4115AA4980D0E3C4CF98, taskId=task_1463520612672_0001_1_00_000000, scheduledTime=1463520612843, launchTime=1463520612843
2016-05-17 21:30:12,843 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520612672_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:12,844 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98]
2016-05-17 21:30:12,844 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [DE5BC638820E47F19404861799CBC5B7/C8FD204DDDBE48609D4E9B30EC4ECD9C] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1, VertexName: B5D2E9C4C0BE4115AA4980D0E3C4CF98, VertexParallelism: 1, TaskAttemptID:attempt_1463520612672_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2278C2920212466F8BFEF5C443937C8F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EA136162D58D4D49BDA3DA1AAA75FF6B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:12,844 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520612672_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:12,844 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:12,845 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520612672_0001, with dag id: dag_1463520612672_0001_1
2016-05-17 21:30:12,845 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:12,846 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:12,846 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:12,846 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520612672_0001_00_000001
2016-05-17 21:30:12,846 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520612672_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:12,848 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520612672_0001_00_000001
2016-05-17 21:30:12,848 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:12,849 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520612672_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:12,850 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:12,850 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612672_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520612672_0001_00_000001, launchTime=1463520612850
2016-05-17 21:30:12,850 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520612672_0001_00_000001
2016-05-17 21:30:12,851 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520612672_0001_1_00_000000_0] to container: [container_1463520612672_0001_00_000001]
2016-05-17 21:30:12,851 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520612672_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:12,851 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520612672_0001_00_000001 given task: attempt_1463520612672_0001_1_00_000000_0
2016-05-17 21:30:12,851 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520612672_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520612672_0001_1_00_000000_0
2016-05-17 21:30:12,851 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520612672_0001_1_00_000000_0] started. Is using containerId: [container_1463520612672_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:12,852 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:12,852 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612672_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B5D2E9C4C0BE4115AA4980D0E3C4CF98, taskAttemptId=attempt_1463520612672_0001_1_00_000000_0, startTime=1463520612851, containerId=container_1463520612672_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520612672_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:12 Starting to run new task attempt: attempt_1463520612672_0001_1_00_000000_0
2016-05-17 21:30:12,852 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520612672_0001_1_00_000000_0
2016-05-17 21:30:12,852 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520612672_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:12,852 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:12,852 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520612672_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:12,853 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:12,853 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:12,853 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:12,853 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:12,854 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:12,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DE5BC638820E47F19404861799CBC5B7/C8FD204DDDBE48609D4E9B30EC4ECD9C] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1, VertexName: B5D2E9C4C0BE4115AA4980D0E3C4CF98, VertexParallelism: 1, TaskAttemptID:attempt_1463520612672_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2278C2920212466F8BFEF5C443937C8F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EA136162D58D4D49BDA3DA1AAA75FF6B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:12,854 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:12,854 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520612672_0001_1_00_000000_0
2016-05-17 21:30:12,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:12,855 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520612672_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:12,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:12,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2278C2920212466F8BFEF5C443937C8F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:12,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EA136162D58D4D49BDA3DA1AAA75FF6B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:12,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:12,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:12,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:12,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2278C2920212466F8BFEF5C443937C8F
2016-05-17 21:30:12,857 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2278C2920212466F8BFEF5C443937C8F
2016-05-17 21:30:12,858 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EA136162D58D4D49BDA3DA1AAA75FF6B
2016-05-17 21:30:12,858 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:12,858 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: EA136162D58D4D49BDA3DA1AAA75FF6B
2016-05-17 21:30:12,862 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:12,863 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:12,863 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2278C2920212466F8BFEF5C443937C8F
2016-05-17 21:30:12,863 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2278C2920212466F8BFEF5C443937C8F
2016-05-17 21:30:12,869 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:12,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EA136162D58D4D49BDA3DA1AAA75FF6B
2016-05-17 21:30:12,870 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B5D2E9C4C0BE4115AA4980D0E3C4CF98, ordinal: 0
2016-05-17 21:30:12,870 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B5D2E9C4C0BE4115AA4980D0E3C4CF98, mem on start (mb), free: 268, total: 383, max: 672
2016-05-17 21:30:12,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:12,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:12,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:12,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:12,871 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:12,871 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:12,871 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:12,871 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:12,872 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:12,872 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:12,872 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:12,872 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2278C2920212466F8BFEF5C443937C8F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:12,873 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EA136162D58D4D49BDA3DA1AAA75FF6B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:12,873 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:12,873 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2278C2920212466F8BFEF5C443937C8F
2016-05-17 21:30:12,873 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:12,873 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2278C2920212466F8BFEF5C443937C8F
2016-05-17 21:30:12,874 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2278C2920212466F8BFEF5C443937C8F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:12,874 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:12,874 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:12,874 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:12,875 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520612672_0001_1_00_000000_0
2016-05-17 21:30:12,876 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:12,877 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6f27c105
2016-05-17 21:30:12,878 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:12,878 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6f27c105
2016-05-17 21:30:12,878 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2278C2920212466F8BFEF5C443937C8F
2016-05-17 21:30:12,879 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1"], id: EA136162D58D4D49BDA3DA1AAA75FF6B
2016-05-17 21:30:12,878 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:12,879 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:12,879 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1"]
2016-05-17 21:30:12,879 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:12,880 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:12,880 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B5D2E9C4C0BE4115AA4980D0E3C4CF98, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:12,883 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:12,883 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:12,884 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520612672_0001_1_00_000000_0
2016-05-17 21:30:12,884 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520612672_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:12,884 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520612672_0001_1_00_000000_0
2016-05-17 21:30:12,885 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B5D2E9C4C0BE4115AA4980D0E3C4CF98, mem on close (mb), free: 266, total: 383, max: 672
2016-05-17 21:30:12 Completed running task attempt: attempt_1463520612672_0001_1_00_000000_0
2016-05-17 21:30:12,886 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520612672_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:12,889 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:12,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612672_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B5D2E9C4C0BE4115AA4980D0E3C4CF98, taskAttemptId=attempt_1463520612672_0001_1_00_000000_0, startTime=1463520612851, finishTime=1463520612889, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520612880, Process_Duration=4, Process_End_Time=1463520612884, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:12,889 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520612672_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:12,890 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520612672_0001_1_00_000000_0
2016-05-17 21:30:12,890 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:12,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:12,891 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612672_0001_1][Event:TASK_FINISHED]: vertexName=B5D2E9C4C0BE4115AA4980D0E3C4CF98, taskId=task_1463520612672_0001_1_00_000000, startTime=1463520612851, finishTime=1463520612890, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1463520612672_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520612880, Process_Duration=4, Process_End_Time=1463520612884, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:12,891 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520612672_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:12,891 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98] : 1
2016-05-17 21:30:12,891 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98]
2016-05-17 21:30:12,892 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:12,892 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612672_0001_1][Event:VERTEX_FINISHED]: vertexName=B5D2E9C4C0BE4115AA4980D0E3C4CF98, vertexId=vertex_1463520612672_0001_1_00, initRequestedTime=1463520612830, initedTime=1463520612842, startRequestedTime=1463520612842, startedTime=1463520612842, finishTime=1463520612891, timeTaken=49, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520612880, Process_Duration=4, Process_End_Time=1463520612884, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520612851, firstTasksToStart=[ task_1463520612672_0001_1_00_000000 ], lastTaskFinishTime=1463520612889, lastTasksToFinish=[ task_1463520612672_0001_1_00_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520612672_0001_1_00_000000 ], longestDurationTasks=[ task_1463520612672_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:12,892 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:12,893 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520612672_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:12,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612672_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520612672_0001_00_000001, stoppedTime=1463520612893, exitStatus=0
2016-05-17 21:30:12,893 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:12,893 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520612672_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:12,893 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:12,893 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520612672_0001_1
2016-05-17 21:30:12,894 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612672_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520612672_0001_1
2016-05-17 21:30:12,894 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: EA136162D58D4D49BDA3DA1AAA75FF6B for vertex: vertex_1463520612672_0001_1_00 [B5D2E9C4C0BE4115AA4980D0E3C4CF98]
2016-05-17 21:30:12,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520612880, Process_Duration=4, Process_End_Time=1463520612884, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:12,895 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:12,895 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:12,896 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520612672_0001_00_000001
2016-05-17 21:30:12,896 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520612672_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:12,897 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520612672_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:12,897 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520612672_0001_00_000001, Exiting
2016-05-17 21:30:12,897 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520612672_0001_00_000001
2016-05-17 21:30:12,897 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520612672_0001_00_000001
2016-05-17 21:30:12,898 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520612672_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:12,898 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520612672_0001_00_000001 completed successfully
2016-05-17 21:30:12,903 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520612672_0001_1, queueSize=0
2016-05-17 21:30:12,903 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:12,904 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520612672_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520612672_0001_1, startTime=1463520612829, finishTime=1463520612902, timeTaken=73, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520612880, Process_Duration=4, Process_End_Time=1463520612884, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:12,904 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520612672_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:12,905 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520612672_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:12,905 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520612672_0001_00_000001
2016-05-17 21:30:12,905 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520612672_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:12 Completed Dag: dag_1463520612672_0001_1
2016-05-17 21:30:12,906 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520612672_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:12,907 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520612672_0001_1_post
2016-05-17 21:30:12,907 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:12,907 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:12,911 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...'_RESULTS.ALL_'field2'_-1, applicationId=application_1463520612672_0001
2016-05-17 21:30:12,913 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:12,913 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:12,913 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:12,913 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:12,913 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:12,914 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:12,914 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:12,915 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:12,915 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:12,915 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:12,916 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:12,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:12,917 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:12,920 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DE5BC638820E47F19404861799CBC5B7/C8FD204DDDBE48609D4E9B30EC4ECD9C/.tez/application_1463520612672_0001
2016-05-17 21:30:12,921 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1/_temporary
2016-05-17 21:30:12,921 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.269
2016-05-17 21:30:12,983 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:12,983 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:12,983 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_-1
2016-05-17 21:30:13,003 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:13,003 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:13,030 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 21:30:13,036 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 21:30:13,037 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:13,037 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:13,037 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:13,049 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:13,049 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'
2016-05-17 21:30:13,057 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:13,058 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:13,058 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'"]
2016-05-17 21:30:13,058 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:13,058 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:13,058 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:13,058 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...LTS.ALL_'field2'_'field2'
2016-05-17 21:30:13,058 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:13,069 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:13,073 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:13,074 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:13,074 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:13,083 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C2CAD3D87F9D4E7F90D3437787658C0E/B4E8DB788B1C4625B76FD17EA3039D88/.tez/application_1463520613073_0001 doesn't exist and is created
2016-05-17 21:30:13,097 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:13,098 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:13,099 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C2CAD3D87F9D4E7F90D3437787658C0E/B4E8DB788B1C4625B76FD17EA3039D88/.tez/application_1463520613073_0001
2016-05-17 21:30:13,101 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520613073_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:13,111 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:13,112 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:13,113 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:13,114 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:13,114 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C2CAD3D87F9D4E7F90D3437787658C0E/B4E8DB788B1C4625B76FD17EA3039D88/.tez/application_1463520613073_0001/localmode-log-dir
2016-05-17 21:30:13,115 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C2CAD3D87F9D4E7F90D3437787658C0E/B4E8DB788B1C4625B76FD17EA3039D88/.tez/application_1463520613073_0001/localmode-log-dir/history.txt.appattempt_1463520613073_0001_000000, maxErrors=10
2016-05-17 21:30:13,115 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:13,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520613073_0001_000000, appSubmitTime=1463520613101, launchTime=1463520613101
2016-05-17 21:30:13,120 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58746
2016-05-17 21:30:13,123 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:13,127 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:13,127 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:13,130 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:13,131 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:13,132 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520613073_0001_000000, startTime=1463520613132
2016-05-17 21:30:13,132 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:13,132 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:13,198 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:13,198 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:13,198 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...LTS.ALL_'field2'_'field2', applicationId=application_1463520613073_0001, dagName=[C2CAD3D87F9D4E7F90D3437787658C0E/B4E8DB788B1C4625B76FD17EA3039D88] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-17 21:30:13,212 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C2CAD3D87F9D4E7F90D3437787658C0E/B4E8DB788B1C4625B76FD17EA3039D88] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-17 21:30:13,214 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520613073_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C2CAD3D87F9D4E7F90D3437787658C0E/B4E8DB788B1C4625B76FD17EA3039D88/.tez/application_1463520613073_0001/localmode-log-dir/dag_1463520613073_0001_1.dot
2016-05-17 21:30:13,214 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C2CAD3D87F9D4E7F90D3437787658C0E/B4E8DB788B1C4625B76FD17EA3039D88/.tez/application_1463520613073_0001/localmode-log-dir/dag_1463520613073_0001_1-tez-dag.pb.txt
2016-05-17 21:30:13,218 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520613073_0001_1
2016-05-17 21:30:13,218 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:13,218 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C2CAD3D87F9D4E7F90D3437787658C0E/B4E8DB788B1C4625B76FD17EA3039D88] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-17 21:30:13 Running Dag: dag_1463520613073_0001_1
2016-05-17 21:30:13,228 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:13,228 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613073_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520613073_0001_1, submitTime=1463520613213
2016-05-17 21:30:13,229 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C0A8227C1031472FB89FBD8B52B684F8
2016-05-17 21:30:13,229 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:13,229 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:13,229 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613073_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520613073_0001_1, initTime=1463520613228
2016-05-17 21:30:13,229 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520613073_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:13,230 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613073_0001_1][Event:DAG_STARTED]: dagID=dag_1463520613073_0001_1, startTime=1463520613230
2016-05-17 21:30:13,230 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:13,230 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520613073_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:13,230 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C0A8227C1031472FB89FBD8B52B684F8 : {5DE45AB25EE142F79A7EB61707BF4530={InputName=5DE45AB25EE142F79A7EB61707BF4530}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:13,230 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 5DE45AB25EE142F79A7EB61707BF4530, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:13,230 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8]
2016-05-17 21:30:13,231 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8]
2016-05-17 21:30:13,231 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8]
2016-05-17 21:30:13,231 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:13,231 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8]. Starting root input initializers: 1
2016-05-17 21:30:13,231 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:13,232 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...LTS.ALL_'field2'_'field2', applicationId=application_1463520613073_0001, dagName=[C2CAD3D87F9D4E7F90D3437787658C0E/B4E8DB788B1C4625B76FD17EA3039D88] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-17 21:30:13,232 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 5DE45AB25EE142F79A7EB61707BF4530 on vertex vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8]
2016-05-17 21:30:13,233 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:13,234 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 5DE45AB25EE142F79A7EB61707BF4530 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:13,234 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:13,234 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:13,235 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:13,236 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 5DE45AB25EE142F79A7EB61707BF4530 on vertex vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8]
2016-05-17 21:30:13,236 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:13,236 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {5DE45AB25EE142F79A7EB61707BF4530=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:13,236 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8] parallelism set to 1
2016-05-17 21:30:13,236 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:13,237 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613073_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520613073_0001_1_00, eventCount=1
2016-05-17 21:30:13,237 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8]
2016-05-17 21:30:13,237 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=162007787C7441FB9D9C13F1170214C2, vertexId=vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:13,237 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=162007787C7441FB9D9C13F1170214C2, vertexId=vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8]
2016-05-17 21:30:13,239 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=162007787C7441FB9D9C13F1170214C2, vertexName=C0A8227C1031472FB89FBD8B52B684F8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:13,239 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=162007787C7441FB9D9C13F1170214C2, vertexId=vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8]
2016-05-17 21:30:13,240 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:13,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613073_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C0A8227C1031472FB89FBD8B52B684F8, vertexId=vertex_1463520613073_0001_1_00, initRequestedTime=1463520613230, initedTime=1463520613240, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:13,240 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8] with distanceFromRoot: 0
2016-05-17 21:30:13,241 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:13,241 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C0A8227C1031472FB89FBD8B52B684F8
2016-05-17 21:30:13,241 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8]
2016-05-17 21:30:13,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:13,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613073_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520613073_0001_1_00, startRequestedTime=1463520613241, startedTime=1463520613241
2016-05-17 21:30:13,242 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:13,242 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:13,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613073_0001_1][Event:TASK_STARTED]: vertexName=C0A8227C1031472FB89FBD8B52B684F8, taskId=task_1463520613073_0001_1_00_000000, scheduledTime=1463520613242, launchTime=1463520613242
2016-05-17 21:30:13,242 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520613073_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:13,243 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8]
2016-05-17 21:30:13,243 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C2CAD3D87F9D4E7F90D3437787658C0E/B4E8DB788B1C4625B76FD17EA3039D88] every.every/(1/1) ...LTS.ALL_'field2'_'field2', VertexName: C0A8227C1031472FB89FBD8B52B684F8, VertexParallelism: 1, TaskAttemptID:attempt_1463520613073_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5DE45AB25EE142F79A7EB61707BF4530, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=162007787C7441FB9D9C13F1170214C2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:13,243 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520613073_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:13,243 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:13,244 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:13,244 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:13,244 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:13,245 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520613073_0001_00_000001
2016-05-17 21:30:13,245 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613073_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:13,247 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520613073_0001_00_000001
2016-05-17 21:30:13,247 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:13,248 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613073_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:13,248 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:13,248 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613073_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520613073_0001_00_000001, launchTime=1463520613248
2016-05-17 21:30:13,248 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520613073_0001, with dag id: dag_1463520613073_0001_1
2016-05-17 21:30:13,249 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520613073_0001_00_000001
2016-05-17 21:30:13,251 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520613073_0001_1_00_000000_0] to container: [container_1463520613073_0001_00_000001]
2016-05-17 21:30:13,251 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613073_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:13,251 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520613073_0001_1_00_000000_0] started. Is using containerId: [container_1463520613073_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:13,251 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613073_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C0A8227C1031472FB89FBD8B52B684F8, taskAttemptId=attempt_1463520613073_0001_1_00_000000_0, startTime=1463520613251, containerId=container_1463520613073_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520613073_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:13,252 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520613073_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:13,252 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520613073_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:13,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:13,252 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520613073_0001_00_000001 given task: attempt_1463520613073_0001_1_00_000000_0
2016-05-17 21:30:13,252 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520613073_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520613073_0001_1_00_000000_0
2016-05-17 21:30:13 Starting to run new task attempt: attempt_1463520613073_0001_1_00_000000_0
2016-05-17 21:30:13,253 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520613073_0001_1_00_000000_0
2016-05-17 21:30:13,253 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:13,253 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:13,253 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:13,253 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:13,254 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:13,254 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:13,254 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C2CAD3D87F9D4E7F90D3437787658C0E/B4E8DB788B1C4625B76FD17EA3039D88] every.every/(1/1) ...LTS.ALL_'field2'_'field2', VertexName: C0A8227C1031472FB89FBD8B52B684F8, VertexParallelism: 1, TaskAttemptID:attempt_1463520613073_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5DE45AB25EE142F79A7EB61707BF4530, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=162007787C7441FB9D9C13F1170214C2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:13,254 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:13,262 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520613073_0001_1_00_000000_0
2016-05-17 21:30:13,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:13,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:13,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=5DE45AB25EE142F79A7EB61707BF4530, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:13,264 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:13,264 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 5DE45AB25EE142F79A7EB61707BF4530
2016-05-17 21:30:13,264 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 5DE45AB25EE142F79A7EB61707BF4530
2016-05-17 21:30:13,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:13,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=162007787C7441FB9D9C13F1170214C2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:13,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:13,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 162007787C7441FB9D9C13F1170214C2
2016-05-17 21:30:13,269 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:13,269 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 5DE45AB25EE142F79A7EB61707BF4530
2016-05-17 21:30:13,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 5DE45AB25EE142F79A7EB61707BF4530
2016-05-17 21:30:13,269 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:13,268 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:13,267 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520613073_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:13,270 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 162007787C7441FB9D9C13F1170214C2
2016-05-17 21:30:13,278 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C0A8227C1031472FB89FBD8B52B684F8, ordinal: 0
2016-05-17 21:30:13,278 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C0A8227C1031472FB89FBD8B52B684F8, mem on start (mb), free: 196, total: 383, max: 672
2016-05-17 21:30:13,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:13,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:13,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:13,280 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:13,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 162007787C7441FB9D9C13F1170214C2
2016-05-17 21:30:13,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:13,280 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:13,280 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:13,280 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:13,280 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:13,281 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:13,281 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:13,281 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:13,281 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 5DE45AB25EE142F79A7EB61707BF4530, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:13,281 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 162007787C7441FB9D9C13F1170214C2, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:13,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:13,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 5DE45AB25EE142F79A7EB61707BF4530 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:13,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 5DE45AB25EE142F79A7EB61707BF4530
2016-05-17 21:30:13,282 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:13,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 5DE45AB25EE142F79A7EB61707BF4530
2016-05-17 21:30:13,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:13,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:13,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:13,283 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520613073_0001_1_00_000000_0
2016-05-17 21:30:13,285 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:13,286 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@e8fb7a8
2016-05-17 21:30:13,286 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:13,287 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@e8fb7a8
2016-05-17 21:30:13,287 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:13,287 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:13,287 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 5DE45AB25EE142F79A7EB61707BF4530
2016-05-17 21:30:13,287 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'"], id: 162007787C7441FB9D9C13F1170214C2
2016-05-17 21:30:13,288 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'"]
2016-05-17 21:30:13,288 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:13,288 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:13,288 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C0A8227C1031472FB89FBD8B52B684F8, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:13,293 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:13,293 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:13,293 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520613073_0001_1_00_000000_0
2016-05-17 21:30:13,293 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520613073_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:13,294 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520613073_0001_1_00_000000_0
2016-05-17 21:30:13,294 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C0A8227C1031472FB89FBD8B52B684F8, mem on close (mb), free: 194, total: 383, max: 672
2016-05-17 21:30:13 Completed running task attempt: attempt_1463520613073_0001_1_00_000000_0
2016-05-17 21:30:13,295 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520613073_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:13,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520613289, Process_Duration=5, Process_End_Time=1463520613294, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:13,297 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613073_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C0A8227C1031472FB89FBD8B52B684F8, taskAttemptId=attempt_1463520613073_0001_1_00_000000_0, startTime=1463520613251, finishTime=1463520613296, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520613289, Process_Duration=5, Process_End_Time=1463520613294, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:13,297 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520613073_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:13,297 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:13,297 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:13,298 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:13,296 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:13,297 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520613073_0001_1_00_000000_0
2016-05-17 21:30:13,299 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:13,299 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613073_0001_1][Event:TASK_FINISHED]: vertexName=C0A8227C1031472FB89FBD8B52B684F8, taskId=task_1463520613073_0001_1_00_000000, startTime=1463520613251, finishTime=1463520613299, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1463520613073_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520613289, Process_Duration=5, Process_End_Time=1463520613294, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:13,299 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520613073_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:13,299 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8] : 1
2016-05-17 21:30:13,300 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8]
2016-05-17 21:30:13,300 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520613073_0001_00_000001
2016-05-17 21:30:13,300 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:13,300 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613073_0001_1][Event:VERTEX_FINISHED]: vertexName=C0A8227C1031472FB89FBD8B52B684F8, vertexId=vertex_1463520613073_0001_1_00, initRequestedTime=1463520613230, initedTime=1463520613240, startRequestedTime=1463520613241, startedTime=1463520613241, finishTime=1463520613300, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520613289, Process_Duration=5, Process_End_Time=1463520613294, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520613251, firstTasksToStart=[ task_1463520613073_0001_1_00_000000 ], lastTaskFinishTime=1463520613296, lastTasksToFinish=[ task_1463520613073_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520613073_0001_1_00_000000 ], longestDurationTasks=[ task_1463520613073_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:13,301 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:13,301 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613073_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:13,301 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613073_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520613073_0001_00_000001, stoppedTime=1463520613301, exitStatus=0
2016-05-17 21:30:13,301 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613073_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:13,301 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:13,301 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:13,302 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520613073_0001_1
2016-05-17 21:30:13,302 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613073_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520613073_0001_1
2016-05-17 21:30:13,302 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 162007787C7441FB9D9C13F1170214C2 for vertex: vertex_1463520613073_0001_1_00 [C0A8227C1031472FB89FBD8B52B684F8]
2016-05-17 21:30:13,310 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520613073_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:13,310 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520613073_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:13,310 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520613073_0001_00_000001, Exiting
2016-05-17 21:30:13,311 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520613073_0001_00_000001
2016-05-17 21:30:13,311 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520613073_0001_00_000001
2016-05-17 21:30:13,312 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520613073_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:13,312 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520613073_0001_00_000001 completed successfully
2016-05-17 21:30:13,314 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520613073_0001_1, queueSize=0
2016-05-17 21:30:13,314 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:13,315 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613073_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520613073_0001_1, startTime=1463520613230, finishTime=1463520613313, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520613289, Process_Duration=5, Process_End_Time=1463520613294, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:13,315 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520613073_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:13,316 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520613073_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:13,316 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520613073_0001_00_000001
2016-05-17 21:30:13,316 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613073_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:13 Completed Dag: dag_1463520613073_0001_1
2016-05-17 21:30:13,317 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520613073_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:13,317 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520613073_0001_1_post
2016-05-17 21:30:13,318 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:13,318 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:13,324 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...LTS.ALL_'field2'_'field2', applicationId=application_1463520613073_0001
2016-05-17 21:30:13,326 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:13,326 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:13,326 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:13,326 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:13,326 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:13,327 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:13,327 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:13,327 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:13,327 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:13,327 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:13,328 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:13,328 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:13,330 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:13,333 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C2CAD3D87F9D4E7F90D3437787658C0E/B4E8DB788B1C4625B76FD17EA3039D88/.tez/application_1463520613073_0001
2016-05-17 21:30:13,333 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'/_temporary
2016-05-17 21:30:13,334 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.276
2016-05-17 21:30:13,398 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_'field2'
2016-05-17 21:30:13,417 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:13,417 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:13,444 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 21:30:13,449 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:30:13,449 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:13,449 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:13,449 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:13,461 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:13,461 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'
2016-05-17 21:30:13,468 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:13,468 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:13,470 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:13,470 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:13,471 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'"]
2016-05-17 21:30:13,471 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:13,471 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:13,471 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:13,472 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ULTS.ALL_'field2'_'field'
2016-05-17 21:30:13,472 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:13,483 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:13,488 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:13,488 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:13,488 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:13,498 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/70CD9A1FAF6241CBAA0384E59B670339/E128B667ABC74CDE861B875A55FD500F/.tez/application_1463520613488_0001 doesn't exist and is created
2016-05-17 21:30:13,513 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:13,513 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:13,514 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/70CD9A1FAF6241CBAA0384E59B670339/E128B667ABC74CDE861B875A55FD500F/.tez/application_1463520613488_0001
2016-05-17 21:30:13,516 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520613488_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:13,525 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:13,525 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:13,527 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:13,527 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:13,527 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/70CD9A1FAF6241CBAA0384E59B670339/E128B667ABC74CDE861B875A55FD500F/.tez/application_1463520613488_0001/localmode-log-dir
2016-05-17 21:30:13,528 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/70CD9A1FAF6241CBAA0384E59B670339/E128B667ABC74CDE861B875A55FD500F/.tez/application_1463520613488_0001/localmode-log-dir/history.txt.appattempt_1463520613488_0001_000000, maxErrors=10
2016-05-17 21:30:13,528 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:13,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520613488_0001_000000, appSubmitTime=1463520613515, launchTime=1463520613515
2016-05-17 21:30:13,532 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:36389
2016-05-17 21:30:13,533 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:13,535 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:13,536 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:13,536 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:13,540 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:13,542 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520613488_0001_000000, startTime=1463520613542
2016-05-17 21:30:13,542 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:13,542 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:13,614 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:13,614 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:13,614 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_'field', applicationId=application_1463520613488_0001, dagName=[70CD9A1FAF6241CBAA0384E59B670339/E128B667ABC74CDE861B875A55FD500F] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-17 21:30:13,627 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [70CD9A1FAF6241CBAA0384E59B670339/E128B667ABC74CDE861B875A55FD500F] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-17 21:30:13,629 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520613488_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/70CD9A1FAF6241CBAA0384E59B670339/E128B667ABC74CDE861B875A55FD500F/.tez/application_1463520613488_0001/localmode-log-dir/dag_1463520613488_0001_1.dot
2016-05-17 21:30:13,629 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/70CD9A1FAF6241CBAA0384E59B670339/E128B667ABC74CDE861B875A55FD500F/.tez/application_1463520613488_0001/localmode-log-dir/dag_1463520613488_0001_1-tez-dag.pb.txt
2016-05-17 21:30:13,634 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520613488_0001_1
2016-05-17 21:30:13,634 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:13,634 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [70CD9A1FAF6241CBAA0384E59B670339/E128B667ABC74CDE861B875A55FD500F] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-17 21:30:13 Running Dag: dag_1463520613488_0001_1
2016-05-17 21:30:13,644 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613488_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520613488_0001_1, submitTime=1463520613628
2016-05-17 21:30:13,644 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:13,644 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C7F896A3AE7B4EA1A8645BCED8319BF5
2016-05-17 21:30:13,644 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:13,645 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613488_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520613488_0001_1, initTime=1463520613644
2016-05-17 21:30:13,645 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520613488_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:13,645 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613488_0001_1][Event:DAG_STARTED]: dagID=dag_1463520613488_0001_1, startTime=1463520613645
2016-05-17 21:30:13,645 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:13,645 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:13,645 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520613488_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:13,645 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:13,646 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C7F896A3AE7B4EA1A8645BCED8319BF5 : {18ED67EFDA514E6E968819A718B32ED9={InputName=18ED67EFDA514E6E968819A718B32ED9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:13,645 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_'field', applicationId=application_1463520613488_0001, dagName=[70CD9A1FAF6241CBAA0384E59B670339/E128B667ABC74CDE861B875A55FD500F] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-17 21:30:13,646 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 18ED67EFDA514E6E968819A718B32ED9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:13,646 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5]
2016-05-17 21:30:13,647 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5]
2016-05-17 21:30:13,647 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5]
2016-05-17 21:30:13,647 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5]. Starting root input initializers: 1
2016-05-17 21:30:13,648 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:13,649 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 18ED67EFDA514E6E968819A718B32ED9 on vertex vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5]
2016-05-17 21:30:13,651 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:13,651 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 18ED67EFDA514E6E968819A718B32ED9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:13,652 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:13,652 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:13,654 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:13,654 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 18ED67EFDA514E6E968819A718B32ED9 on vertex vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5]
2016-05-17 21:30:13,655 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:13,655 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {18ED67EFDA514E6E968819A718B32ED9=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:13,655 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5] parallelism set to 1
2016-05-17 21:30:13,655 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:13,655 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613488_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520613488_0001_1_00, eventCount=1
2016-05-17 21:30:13,656 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5]
2016-05-17 21:30:13,656 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0E7D4976555E46EE95A78DA14DF94A54, vertexId=vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:13,656 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0E7D4976555E46EE95A78DA14DF94A54, vertexId=vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5]
2016-05-17 21:30:13,657 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0E7D4976555E46EE95A78DA14DF94A54, vertexName=C7F896A3AE7B4EA1A8645BCED8319BF5, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:13,658 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0E7D4976555E46EE95A78DA14DF94A54, vertexId=vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5]
2016-05-17 21:30:13,658 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520613488_0001, with dag id: dag_1463520613488_0001_1
2016-05-17 21:30:13,659 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613488_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C7F896A3AE7B4EA1A8645BCED8319BF5, vertexId=vertex_1463520613488_0001_1_00, initRequestedTime=1463520613646, initedTime=1463520613659, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:13,659 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:13,659 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5] with distanceFromRoot: 0
2016-05-17 21:30:13,659 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:13,660 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C7F896A3AE7B4EA1A8645BCED8319BF5
2016-05-17 21:30:13,660 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5]
2016-05-17 21:30:13,661 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613488_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520613488_0001_1_00, startRequestedTime=1463520613660, startedTime=1463520613660
2016-05-17 21:30:13,661 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:13,661 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:13,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:13,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613488_0001_1][Event:TASK_STARTED]: vertexName=C7F896A3AE7B4EA1A8645BCED8319BF5, taskId=task_1463520613488_0001_1_00_000000, scheduledTime=1463520613662, launchTime=1463520613662
2016-05-17 21:30:13,662 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520613488_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:13,663 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5]
2016-05-17 21:30:13,663 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [70CD9A1FAF6241CBAA0384E59B670339/E128B667ABC74CDE861B875A55FD500F] every.every/(1/1) ...ULTS.ALL_'field2'_'field', VertexName: C7F896A3AE7B4EA1A8645BCED8319BF5, VertexParallelism: 1, TaskAttemptID:attempt_1463520613488_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=18ED67EFDA514E6E968819A718B32ED9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0E7D4976555E46EE95A78DA14DF94A54, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:13,664 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520613488_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:13,664 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:13,664 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:13,665 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:13,665 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:13,666 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520613488_0001_00_000001
2016-05-17 21:30:13,666 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613488_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:13,668 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520613488_0001_00_000001
2016-05-17 21:30:13,668 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:13,668 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613488_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:13,668 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613488_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520613488_0001_00_000001, launchTime=1463520613668
2016-05-17 21:30:13,668 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:13,669 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520613488_0001_00_000001
2016-05-17 21:30:13,670 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520613488_0001_1_00_000000_0] to container: [container_1463520613488_0001_00_000001]
2016-05-17 21:30:13,670 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613488_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:13,670 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520613488_0001_00_000001 given task: attempt_1463520613488_0001_1_00_000000_0
2016-05-17 21:30:13,670 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520613488_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520613488_0001_1_00_000000_0
2016-05-17 21:30:13,670 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520613488_0001_1_00_000000_0] started. Is using containerId: [container_1463520613488_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:13 Starting to run new task attempt: attempt_1463520613488_0001_1_00_000000_0
2016-05-17 21:30:13,670 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520613488_0001_1_00_000000_0
2016-05-17 21:30:13,671 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:13,671 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:13,671 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:13,671 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:13,671 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:13,671 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:13,671 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613488_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C7F896A3AE7B4EA1A8645BCED8319BF5, taskAttemptId=attempt_1463520613488_0001_1_00_000000_0, startTime=1463520613670, containerId=container_1463520613488_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520613488_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:13,671 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:13,672 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [70CD9A1FAF6241CBAA0384E59B670339/E128B667ABC74CDE861B875A55FD500F] every.every/(1/1) ...ULTS.ALL_'field2'_'field', VertexName: C7F896A3AE7B4EA1A8645BCED8319BF5, VertexParallelism: 1, TaskAttemptID:attempt_1463520613488_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=18ED67EFDA514E6E968819A718B32ED9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0E7D4976555E46EE95A78DA14DF94A54, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:13,672 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520613488_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:13,672 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:13,672 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520613488_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:13,673 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520613488_0001_1_00_000000_0
2016-05-17 21:30:13,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:13,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:13,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=18ED67EFDA514E6E968819A718B32ED9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:13,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:13,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 18ED67EFDA514E6E968819A718B32ED9
2016-05-17 21:30:13,674 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 18ED67EFDA514E6E968819A718B32ED9
2016-05-17 21:30:13,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:13,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0E7D4976555E46EE95A78DA14DF94A54, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:13,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:13,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0E7D4976555E46EE95A78DA14DF94A54
2016-05-17 21:30:13,676 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:13,677 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0E7D4976555E46EE95A78DA14DF94A54
2016-05-17 21:30:13,677 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:13,678 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:13,678 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 18ED67EFDA514E6E968819A718B32ED9
2016-05-17 21:30:13,678 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 18ED67EFDA514E6E968819A718B32ED9
2016-05-17 21:30:13,679 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520613488_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:13,683 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C7F896A3AE7B4EA1A8645BCED8319BF5, ordinal: 0
2016-05-17 21:30:13,683 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C7F896A3AE7B4EA1A8645BCED8319BF5, mem on start (mb), free: 122, total: 383, max: 672
2016-05-17 21:30:13,683 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:13,683 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:13,684 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:13,688 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:13,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0E7D4976555E46EE95A78DA14DF94A54
2016-05-17 21:30:13,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:13,688 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:13,689 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:13,689 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:13,689 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:13,689 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:13,690 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:13,690 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:13,690 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 18ED67EFDA514E6E968819A718B32ED9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:13,690 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0E7D4976555E46EE95A78DA14DF94A54, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:13,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:13,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 18ED67EFDA514E6E968819A718B32ED9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:13,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 18ED67EFDA514E6E968819A718B32ED9
2016-05-17 21:30:13,691 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:13,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 18ED67EFDA514E6E968819A718B32ED9
2016-05-17 21:30:13,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:13,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:13,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:13,692 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520613488_0001_1_00_000000_0
2016-05-17 21:30:13,692 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:13,693 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@365c5ff9
2016-05-17 21:30:13,694 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@365c5ff9
2016-05-17 21:30:13,694 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:13,694 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:13,695 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:13,695 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 18ED67EFDA514E6E968819A718B32ED9
2016-05-17 21:30:13,695 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'"], id: 0E7D4976555E46EE95A78DA14DF94A54
2016-05-17 21:30:13,696 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'"]
2016-05-17 21:30:13,696 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:13,696 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:13,696 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C7F896A3AE7B4EA1A8645BCED8319BF5, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:13,698 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:13,699 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:13,699 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520613488_0001_1_00_000000_0
2016-05-17 21:30:13,699 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520613488_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:13,700 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520613488_0001_1_00_000000_0
2016-05-17 21:30:13,700 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C7F896A3AE7B4EA1A8645BCED8319BF5, mem on close (mb), free: 120, total: 383, max: 672
2016-05-17 21:30:13 Completed running task attempt: attempt_1463520613488_0001_1_00_000000_0
2016-05-17 21:30:13,700 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520613488_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:13,703 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520613696, Process_Duration=3, Process_End_Time=1463520613699, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:13,704 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613488_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C7F896A3AE7B4EA1A8645BCED8319BF5, taskAttemptId=attempt_1463520613488_0001_1_00_000000_0, startTime=1463520613670, finishTime=1463520613703, timeTaken=33, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520613696, Process_Duration=3, Process_End_Time=1463520613699, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:13,704 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:13,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:13,704 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:13,704 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520613488_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:13,704 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:13,704 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520613488_0001_00_000001
2016-05-17 21:30:13,704 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520613488_0001_1_00_000000_0
2016-05-17 21:30:13,705 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:13,705 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613488_0001_1][Event:TASK_FINISHED]: vertexName=C7F896A3AE7B4EA1A8645BCED8319BF5, taskId=task_1463520613488_0001_1_00_000000, startTime=1463520613670, finishTime=1463520613705, timeTaken=35, status=SUCCEEDED, successfulAttemptID=attempt_1463520613488_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520613696, Process_Duration=3, Process_End_Time=1463520613699, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:13,705 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520613488_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:13,705 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5] : 1
2016-05-17 21:30:13,706 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5]
2016-05-17 21:30:13,706 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:13,706 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613488_0001_1][Event:VERTEX_FINISHED]: vertexName=C7F896A3AE7B4EA1A8645BCED8319BF5, vertexId=vertex_1463520613488_0001_1_00, initRequestedTime=1463520613646, initedTime=1463520613659, startRequestedTime=1463520613660, startedTime=1463520613660, finishTime=1463520613706, timeTaken=46, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520613696, Process_Duration=3, Process_End_Time=1463520613699, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520613670, firstTasksToStart=[ task_1463520613488_0001_1_00_000000 ], lastTaskFinishTime=1463520613703, lastTasksToFinish=[ task_1463520613488_0001_1_00_000000 ], minTaskDuration=33, maxTaskDuration=33, avgTaskDuration=33.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520613488_0001_1_00_000000 ], longestDurationTasks=[ task_1463520613488_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:13,706 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:13,706 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613488_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:13,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613488_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520613488_0001_00_000001, stoppedTime=1463520613707, exitStatus=0
2016-05-17 21:30:13,707 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613488_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:13,707 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:13,707 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520613488_0001_1
2016-05-17 21:30:13,707 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:13,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613488_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520613488_0001_1
2016-05-17 21:30:13,708 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0E7D4976555E46EE95A78DA14DF94A54 for vertex: vertex_1463520613488_0001_1_00 [C7F896A3AE7B4EA1A8645BCED8319BF5]
2016-05-17 21:30:13,715 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520613488_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:13,715 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520613488_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:13,715 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520613488_0001_00_000001, Exiting
2016-05-17 21:30:13,715 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520613488_0001_00_000001
2016-05-17 21:30:13,715 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520613488_0001_00_000001
2016-05-17 21:30:13,716 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520613488_0001_1, queueSize=0
2016-05-17 21:30:13,716 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520613488_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:13,716 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:13,716 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520613488_0001_00_000001 completed successfully
2016-05-17 21:30:13,717 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613488_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520613488_0001_1, startTime=1463520613645, finishTime=1463520613714, timeTaken=69, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520613696, Process_Duration=3, Process_End_Time=1463520613699, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:13,717 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520613488_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:13,717 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520613488_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:13,718 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520613488_0001_00_000001
2016-05-17 21:30:13,718 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613488_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:13 Completed Dag: dag_1463520613488_0001_1
2016-05-17 21:30:13,720 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520613488_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:13,720 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520613488_0001_1_post
2016-05-17 21:30:13,721 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:13,721 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:13,723 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ULTS.ALL_'field2'_'field', applicationId=application_1463520613488_0001
2016-05-17 21:30:13,725 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:13,725 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:13,725 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:13,725 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:13,725 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:13,726 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:13,726 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:13,726 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:13,726 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:13,727 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:13,727 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:13,727 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:13,727 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:13,731 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/70CD9A1FAF6241CBAA0384E59B670339/E128B667ABC74CDE861B875A55FD500F/.tez/application_1463520613488_0001
2016-05-17 21:30:13,731 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'/_temporary
2016-05-17 21:30:13,732 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.262
2016-05-17 21:30:13,815 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_'field'
2016-05-17 21:30:13,834 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:13,834 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:13,861 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 21:30:13,866 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:30:13,866 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:13,866 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:13,866 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:13,880 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:13,880 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS
2016-05-17 21:30:13,889 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:13,889 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:13,889 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS"]
2016-05-17 21:30:13,889 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:13,890 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:13,890 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:13,891 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-17 21:30:13,891 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:13,900 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:13,901 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:13,902 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:13,906 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:13,906 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:13,907 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:13,915 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/90380B4565EE4F4D9105A3AF4C216A0C/CCD3D6EF078F415D9176F143810AFED9/.tez/application_1463520613906_0001 doesn't exist and is created
2016-05-17 21:30:13,932 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:13,932 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:13,932 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/90380B4565EE4F4D9105A3AF4C216A0C/CCD3D6EF078F415D9176F143810AFED9/.tez/application_1463520613906_0001
2016-05-17 21:30:13,934 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520613906_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:13,944 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:13,944 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:13,945 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:13,945 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:13,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/90380B4565EE4F4D9105A3AF4C216A0C/CCD3D6EF078F415D9176F143810AFED9/.tez/application_1463520613906_0001/localmode-log-dir
2016-05-17 21:30:13,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/90380B4565EE4F4D9105A3AF4C216A0C/CCD3D6EF078F415D9176F143810AFED9/.tez/application_1463520613906_0001/localmode-log-dir/history.txt.appattempt_1463520613906_0001_000000, maxErrors=10
2016-05-17 21:30:13,946 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:13,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520613906_0001_000000, appSubmitTime=1463520613934, launchTime=1463520613934
2016-05-17 21:30:13,951 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:13,952 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:36534
2016-05-17 21:30:13,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:13,955 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:13,956 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:13,956 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:13,958 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520613906_0001_000000, startTime=1463520613958
2016-05-17 21:30:13,958 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:13,958 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:14,032 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:14,032 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:14,032 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_RESULTS, applicationId=application_1463520613906_0001, dagName=[90380B4565EE4F4D9105A3AF4C216A0C/CCD3D6EF078F415D9176F143810AFED9] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-17 21:30:14,046 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [90380B4565EE4F4D9105A3AF4C216A0C/CCD3D6EF078F415D9176F143810AFED9] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-17 21:30:14,047 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520613906_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/90380B4565EE4F4D9105A3AF4C216A0C/CCD3D6EF078F415D9176F143810AFED9/.tez/application_1463520613906_0001/localmode-log-dir/dag_1463520613906_0001_1.dot
2016-05-17 21:30:14,047 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/90380B4565EE4F4D9105A3AF4C216A0C/CCD3D6EF078F415D9176F143810AFED9/.tez/application_1463520613906_0001/localmode-log-dir/dag_1463520613906_0001_1-tez-dag.pb.txt
2016-05-17 21:30:14,052 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520613906_0001_1
2016-05-17 21:30:14,052 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:14,052 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [90380B4565EE4F4D9105A3AF4C216A0C/CCD3D6EF078F415D9176F143810AFED9] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-17 21:30:14 Running Dag: dag_1463520613906_0001_1
2016-05-17 21:30:14,062 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613906_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520613906_0001_1, submitTime=1463520614046
2016-05-17 21:30:14,062 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:14,062 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DD1F592E6B554214ACAA5D757DD568FA
2016-05-17 21:30:14,063 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:14,063 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613906_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520613906_0001_1, initTime=1463520614062
2016-05-17 21:30:14,063 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:14,063 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520613906_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:14,064 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613906_0001_1][Event:DAG_STARTED]: dagID=dag_1463520613906_0001_1, startTime=1463520614064
2016-05-17 21:30:14,064 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:14,064 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520613906_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:14,064 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DD1F592E6B554214ACAA5D757DD568FA : {86B40AE196BD4924A0A224ED9161D018={InputName=86B40AE196BD4924A0A224ED9161D018}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:14,064 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 86B40AE196BD4924A0A224ED9161D018, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:14,064 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA]
2016-05-17 21:30:14,065 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA]
2016-05-17 21:30:14,065 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA]
2016-05-17 21:30:14,065 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA]. Starting root input initializers: 1
2016-05-17 21:30:14,064 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:14,065 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_RESULTS, applicationId=application_1463520613906_0001, dagName=[90380B4565EE4F4D9105A3AF4C216A0C/CCD3D6EF078F415D9176F143810AFED9] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-17 21:30:14,065 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:14,066 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 86B40AE196BD4924A0A224ED9161D018 on vertex vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA]
2016-05-17 21:30:14,067 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:14,067 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 86B40AE196BD4924A0A224ED9161D018 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:14,068 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:14,068 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:14,069 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:14,070 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 86B40AE196BD4924A0A224ED9161D018 on vertex vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA]
2016-05-17 21:30:14,070 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:14,070 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {86B40AE196BD4924A0A224ED9161D018=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:14,070 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA] parallelism set to 1
2016-05-17 21:30:14,070 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:14,071 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613906_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520613906_0001_1_00, eventCount=1
2016-05-17 21:30:14,071 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA]
2016-05-17 21:30:14,071 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=225A0F9929A04EB19F6AD9E9A4FFC8B0, vertexId=vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:14,071 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=225A0F9929A04EB19F6AD9E9A4FFC8B0, vertexId=vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA]
2016-05-17 21:30:14,073 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=225A0F9929A04EB19F6AD9E9A4FFC8B0, vertexName=DD1F592E6B554214ACAA5D757DD568FA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:14,073 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=225A0F9929A04EB19F6AD9E9A4FFC8B0, vertexId=vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA]
2016-05-17 21:30:14,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613906_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DD1F592E6B554214ACAA5D757DD568FA, vertexId=vertex_1463520613906_0001_1_00, initRequestedTime=1463520614064, initedTime=1463520614074, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:14,074 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA] with distanceFromRoot: 0
2016-05-17 21:30:14,074 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:14,074 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:14,074 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DD1F592E6B554214ACAA5D757DD568FA
2016-05-17 21:30:14,074 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA]
2016-05-17 21:30:14,075 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613906_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520613906_0001_1_00, startRequestedTime=1463520614074, startedTime=1463520614074
2016-05-17 21:30:14,075 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:14,075 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:14,075 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:14,076 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613906_0001_1][Event:TASK_STARTED]: vertexName=DD1F592E6B554214ACAA5D757DD568FA, taskId=task_1463520613906_0001_1_00_000000, scheduledTime=1463520614075, launchTime=1463520614075
2016-05-17 21:30:14,076 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520613906_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:14,076 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA]
2016-05-17 21:30:14,076 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [90380B4565EE4F4D9105A3AF4C216A0C/CCD3D6EF078F415D9176F143810AFED9] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS, VertexName: DD1F592E6B554214ACAA5D757DD568FA, VertexParallelism: 1, TaskAttemptID:attempt_1463520613906_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=86B40AE196BD4924A0A224ED9161D018, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=225A0F9929A04EB19F6AD9E9A4FFC8B0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:14,076 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520613906_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:14,076 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:14,077 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:14,077 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:14,077 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:14,078 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520613906_0001_00_000001
2016-05-17 21:30:14,078 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613906_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:14,079 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520613906_0001_00_000001
2016-05-17 21:30:14,080 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:14,081 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520613906_0001, with dag id: dag_1463520613906_0001_1
2016-05-17 21:30:14,083 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613906_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:14,083 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613906_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520613906_0001_00_000001, launchTime=1463520614082
2016-05-17 21:30:14,083 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:14,084 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520613906_0001_00_000001
2016-05-17 21:30:14,085 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520613906_0001_1_00_000000_0] to container: [container_1463520613906_0001_00_000001]
2016-05-17 21:30:14,085 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613906_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:14,086 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520613906_0001_1_00_000000_0] started. Is using containerId: [container_1463520613906_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:14,086 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613906_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DD1F592E6B554214ACAA5D757DD568FA, taskAttemptId=attempt_1463520613906_0001_1_00_000000_0, startTime=1463520614086, containerId=container_1463520613906_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520613906_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:14,086 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520613906_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:14,086 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520613906_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:14,086 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:14,087 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520613906_0001_00_000001 given task: attempt_1463520613906_0001_1_00_000000_0
2016-05-17 21:30:14,087 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520613906_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520613906_0001_1_00_000000_0
2016-05-17 21:30:14 Starting to run new task attempt: attempt_1463520613906_0001_1_00_000000_0
2016-05-17 21:30:14,088 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520613906_0001_1_00_000000_0
2016-05-17 21:30:14,088 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:14,088 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:14,088 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:14,088 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:14,088 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:14,088 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:14,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [90380B4565EE4F4D9105A3AF4C216A0C/CCD3D6EF078F415D9176F143810AFED9] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS, VertexName: DD1F592E6B554214ACAA5D757DD568FA, VertexParallelism: 1, TaskAttemptID:attempt_1463520613906_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=86B40AE196BD4924A0A224ED9161D018, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=225A0F9929A04EB19F6AD9E9A4FFC8B0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:14,089 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:14,090 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520613906_0001_1_00_000000_0
2016-05-17 21:30:14,090 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:14,090 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:14,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:14,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=86B40AE196BD4924A0A224ED9161D018, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:14,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:14,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 86B40AE196BD4924A0A224ED9161D018
2016-05-17 21:30:14,095 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 86B40AE196BD4924A0A224ED9161D018
2016-05-17 21:30:14,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=225A0F9929A04EB19F6AD9E9A4FFC8B0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:14,093 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520613906_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:14,096 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:14,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:14,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 225A0F9929A04EB19F6AD9E9A4FFC8B0
2016-05-17 21:30:14,097 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:14,097 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 225A0F9929A04EB19F6AD9E9A4FFC8B0
2016-05-17 21:30:14,098 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:14,100 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 86B40AE196BD4924A0A224ED9161D018
2016-05-17 21:30:14,101 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 86B40AE196BD4924A0A224ED9161D018
2016-05-17 21:30:14,104 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DD1F592E6B554214ACAA5D757DD568FA, ordinal: 0
2016-05-17 21:30:14,104 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD1F592E6B554214ACAA5D757DD568FA, mem on start (mb), free: 233, total: 364, max: 672
2016-05-17 21:30:14,104 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:14,105 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:14,105 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:14,110 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:14,110 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 225A0F9929A04EB19F6AD9E9A4FFC8B0
2016-05-17 21:30:14,110 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:14,110 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:14,111 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:14,111 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:14,111 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:14,112 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:14,112 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:14,112 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:14,113 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 86B40AE196BD4924A0A224ED9161D018, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:14,113 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 225A0F9929A04EB19F6AD9E9A4FFC8B0, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:14,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:14,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 86B40AE196BD4924A0A224ED9161D018
2016-05-17 21:30:14,114 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:14,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 86B40AE196BD4924A0A224ED9161D018
2016-05-17 21:30:14,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 86B40AE196BD4924A0A224ED9161D018 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:14,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:14,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:14,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:14,116 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520613906_0001_1_00_000000_0
2016-05-17 21:30:14,116 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:14,117 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3faaac68
2016-05-17 21:30:14,118 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3faaac68
2016-05-17 21:30:14,118 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:14,118 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:14,120 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:14,121 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 86B40AE196BD4924A0A224ED9161D018
2016-05-17 21:30:14,121 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS"], id: 225A0F9929A04EB19F6AD9E9A4FFC8B0
2016-05-17 21:30:14,122 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS"]
2016-05-17 21:30:14,122 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:14,122 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:14,123 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DD1F592E6B554214ACAA5D757DD568FA, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:14,126 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:14,126 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:14,126 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520613906_0001_1_00_000000_0
2016-05-17 21:30:14,127 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520613906_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:14,128 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520613906_0001_1_00_000000_0
2016-05-17 21:30:14,128 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD1F592E6B554214ACAA5D757DD568FA, mem on close (mb), free: 231, total: 364, max: 672
2016-05-17 21:30:14 Completed running task attempt: attempt_1463520613906_0001_1_00_000000_0
2016-05-17 21:30:14,129 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520613906_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:14,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520614123, Process_Duration=5, Process_End_Time=1463520614128, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:14,130 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:14,130 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613906_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DD1F592E6B554214ACAA5D757DD568FA, taskAttemptId=attempt_1463520613906_0001_1_00_000000_0, startTime=1463520614086, finishTime=1463520614130, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520614123, Process_Duration=5, Process_End_Time=1463520614128, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:14,131 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:14,131 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520613906_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:14,131 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:14,131 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520613906_0001_1_00_000000_0
2016-05-17 21:30:14,131 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:14,131 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520613906_0001_00_000001
2016-05-17 21:30:14,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613906_0001_1][Event:TASK_FINISHED]: vertexName=DD1F592E6B554214ACAA5D757DD568FA, taskId=task_1463520613906_0001_1_00_000000, startTime=1463520614086, finishTime=1463520614131, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1463520613906_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520614123, Process_Duration=5, Process_End_Time=1463520614128, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:14,135 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520613906_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:14,135 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA] : 1
2016-05-17 21:30:14,135 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA]
2016-05-17 21:30:14,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613906_0001_1][Event:VERTEX_FINISHED]: vertexName=DD1F592E6B554214ACAA5D757DD568FA, vertexId=vertex_1463520613906_0001_1_00, initRequestedTime=1463520614064, initedTime=1463520614074, startRequestedTime=1463520614074, startedTime=1463520614074, finishTime=1463520614135, timeTaken=61, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520614123, Process_Duration=5, Process_End_Time=1463520614128, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520614086, firstTasksToStart=[ task_1463520613906_0001_1_00_000000 ], lastTaskFinishTime=1463520614130, lastTasksToFinish=[ task_1463520613906_0001_1_00_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520613906_0001_1_00_000000 ], longestDurationTasks=[ task_1463520613906_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:14,136 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:14,134 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:14,136 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613906_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:14,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613906_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520613906_0001_00_000001, stoppedTime=1463520614136, exitStatus=0
2016-05-17 21:30:14,137 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613906_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:14,137 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:14,137 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:14,137 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520613906_0001_1
2016-05-17 21:30:14,137 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:14,137 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613906_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520613906_0001_1
2016-05-17 21:30:14,137 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 225A0F9929A04EB19F6AD9E9A4FFC8B0 for vertex: vertex_1463520613906_0001_1_00 [DD1F592E6B554214ACAA5D757DD568FA]
2016-05-17 21:30:14,145 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520613906_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:14,145 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520613906_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:14,145 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520613906_0001_00_000001, Exiting
2016-05-17 21:30:14,145 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520613906_0001_00_000001
2016-05-17 21:30:14,145 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520613906_0001_00_000001
2016-05-17 21:30:14,146 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520613906_0001_1, queueSize=0
2016-05-17 21:30:14,146 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520613906_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:14,146 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:14,146 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520613906_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520613906_0001_1, startTime=1463520614064, finishTime=1463520614145, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520614123, Process_Duration=5, Process_End_Time=1463520614128, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:14,147 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520613906_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:14,147 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520613906_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:14,147 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520613906_0001_00_000001
2016-05-17 21:30:14 Completed Dag: dag_1463520613906_0001_1
2016-05-17 21:30:14,147 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520613906_0001_00_000001 completed successfully
2016-05-17 21:30:14,147 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520613906_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:14,148 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520613906_0001_1_post
2016-05-17 21:30:14,148 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:14,148 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:14,148 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613906_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:30:14,148 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520613906_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:30:14,153 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ULTS.ALL_'field2'_RESULTS, applicationId=application_1463520613906_0001
2016-05-17 21:30:14,155 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:14,155 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:14,155 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:14,155 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:14,155 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:14,156 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:14,156 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:14,156 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:14,156 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:14,157 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:14,157 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:14,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:14,158 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:14,163 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/90380B4565EE4F4D9105A3AF4C216A0C/CCD3D6EF078F415D9176F143810AFED9/.tez/application_1463520613906_0001
2016-05-17 21:30:14,163 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS/_temporary
2016-05-17 21:30:14,163 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.274
2016-05-17 21:30:14,225 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_RESULTS
2016-05-17 21:30:14,244 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:14,245 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:14,274 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 21:30:14,282 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 21:30:14,282 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:14,283 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:14,283 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:14,298 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:14,298 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL
2016-05-17 21:30:14,307 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:14,307 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:14,307 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL"]
2016-05-17 21:30:14,307 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:14,307 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:14,307 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:14,308 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-17 21:30:14,308 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:14,320 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:14,324 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:14,324 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:14,324 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:14,333 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/92F6A28CBB38435D9E30BCA436212BEA/E707E93543714D2F80A78529824771AD/.tez/application_1463520614324_0001 doesn't exist and is created
2016-05-17 21:30:14,348 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:14,348 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:14,348 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/92F6A28CBB38435D9E30BCA436212BEA/E707E93543714D2F80A78529824771AD/.tez/application_1463520614324_0001
2016-05-17 21:30:14,351 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520614324_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:14,360 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:14,361 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:14,363 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:14,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:14,363 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/92F6A28CBB38435D9E30BCA436212BEA/E707E93543714D2F80A78529824771AD/.tez/application_1463520614324_0001/localmode-log-dir
2016-05-17 21:30:14,363 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/92F6A28CBB38435D9E30BCA436212BEA/E707E93543714D2F80A78529824771AD/.tez/application_1463520614324_0001/localmode-log-dir/history.txt.appattempt_1463520614324_0001_000000, maxErrors=10
2016-05-17 21:30:14,363 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:14,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520614324_0001_000000, appSubmitTime=1463520614350, launchTime=1463520614351
2016-05-17 21:30:14,369 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:14,369 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:14,370 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:14,370 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:14,373 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:44693
2016-05-17 21:30:14,376 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:14,378 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520614324_0001_000000, startTime=1463520614378
2016-05-17 21:30:14,378 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:14,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:14,406 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:14,406 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:14,448 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:14,448 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:14,448 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ..._RESULTS.ALL_'field2'_ALL, applicationId=application_1463520614324_0001, dagName=[92F6A28CBB38435D9E30BCA436212BEA/E707E93543714D2F80A78529824771AD] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-17 21:30:14,462 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [92F6A28CBB38435D9E30BCA436212BEA/E707E93543714D2F80A78529824771AD] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-17 21:30:14,463 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520614324_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/92F6A28CBB38435D9E30BCA436212BEA/E707E93543714D2F80A78529824771AD/.tez/application_1463520614324_0001/localmode-log-dir/dag_1463520614324_0001_1.dot
2016-05-17 21:30:14,464 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/92F6A28CBB38435D9E30BCA436212BEA/E707E93543714D2F80A78529824771AD/.tez/application_1463520614324_0001/localmode-log-dir/dag_1463520614324_0001_1-tez-dag.pb.txt
2016-05-17 21:30:14,467 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520614324_0001_1
2016-05-17 21:30:14,467 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:14,467 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [92F6A28CBB38435D9E30BCA436212BEA/E707E93543714D2F80A78529824771AD] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-17 21:30:14 Running Dag: dag_1463520614324_0001_1
2016-05-17 21:30:14,477 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614324_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520614324_0001_1, submitTime=1463520614462
2016-05-17 21:30:14,477 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:14,478 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CAB878E189D3428A9DA5770B08519658
2016-05-17 21:30:14,478 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:14,479 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614324_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520614324_0001_1, initTime=1463520614478
2016-05-17 21:30:14,479 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:14,479 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520614324_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:14,480 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614324_0001_1][Event:DAG_STARTED]: dagID=dag_1463520614324_0001_1, startTime=1463520614480
2016-05-17 21:30:14,480 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:14,481 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520614324_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:14,481 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CAB878E189D3428A9DA5770B08519658 : {6EFF6FE19A9345D8858C198B870E0A42={InputName=6EFF6FE19A9345D8858C198B870E0A42}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:14,481 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 6EFF6FE19A9345D8858C198B870E0A42, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:14,481 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658]
2016-05-17 21:30:14,481 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ..._RESULTS.ALL_'field2'_ALL, applicationId=application_1463520614324_0001, dagName=[92F6A28CBB38435D9E30BCA436212BEA/E707E93543714D2F80A78529824771AD] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-17 21:30:14,481 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:14,481 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658]
2016-05-17 21:30:14,481 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658]
2016-05-17 21:30:14,481 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658]. Starting root input initializers: 1
2016-05-17 21:30:14,482 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:14,482 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 6EFF6FE19A9345D8858C198B870E0A42 on vertex vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658]
2016-05-17 21:30:14,484 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:14,485 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 6EFF6FE19A9345D8858C198B870E0A42 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:14,485 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:14,486 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:14,487 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:14,487 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 6EFF6FE19A9345D8858C198B870E0A42 on vertex vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658]
2016-05-17 21:30:14,487 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:14,487 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {6EFF6FE19A9345D8858C198B870E0A42=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:14,488 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658] parallelism set to 1
2016-05-17 21:30:14,488 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:14,488 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614324_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520614324_0001_1_00, eventCount=1
2016-05-17 21:30:14,488 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658]
2016-05-17 21:30:14,488 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=5667E6C4803C4D278A143776703FE96A, vertexId=vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:14,488 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=5667E6C4803C4D278A143776703FE96A, vertexId=vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658]
2016-05-17 21:30:14,490 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=5667E6C4803C4D278A143776703FE96A, vertexName=CAB878E189D3428A9DA5770B08519658, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:14,490 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=5667E6C4803C4D278A143776703FE96A, vertexId=vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658]
2016-05-17 21:30:14,491 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:14,491 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614324_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CAB878E189D3428A9DA5770B08519658, vertexId=vertex_1463520614324_0001_1_00, initRequestedTime=1463520614481, initedTime=1463520614491, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:14,492 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658] with distanceFromRoot: 0
2016-05-17 21:30:14,492 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:14,492 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CAB878E189D3428A9DA5770B08519658
2016-05-17 21:30:14,492 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658]
2016-05-17 21:30:14,493 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:14,493 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614324_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520614324_0001_1_00, startRequestedTime=1463520614492, startedTime=1463520614492
2016-05-17 21:30:14,493 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:14,494 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:14,494 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614324_0001_1][Event:TASK_STARTED]: vertexName=CAB878E189D3428A9DA5770B08519658, taskId=task_1463520614324_0001_1_00_000000, scheduledTime=1463520614494, launchTime=1463520614494
2016-05-17 21:30:14,494 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520614324_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:14,494 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658]
2016-05-17 21:30:14,495 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [92F6A28CBB38435D9E30BCA436212BEA/E707E93543714D2F80A78529824771AD] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL, VertexName: CAB878E189D3428A9DA5770B08519658, VertexParallelism: 1, TaskAttemptID:attempt_1463520614324_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6EFF6FE19A9345D8858C198B870E0A42, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5667E6C4803C4D278A143776703FE96A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:14,495 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520614324_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:14,495 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:14,496 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:14,497 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:14,497 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:14,497 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520614324_0001_00_000001
2016-05-17 21:30:14,497 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520614324_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:14,499 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520614324_0001_00_000001
2016-05-17 21:30:14,499 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:14,500 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520614324_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:14,500 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:14,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614324_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520614324_0001_00_000001, launchTime=1463520614500
2016-05-17 21:30:14,501 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520614324_0001_00_000001
2016-05-17 21:30:14,502 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520614324_0001_1_00_000000_0] to container: [container_1463520614324_0001_00_000001]
2016-05-17 21:30:14,502 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520614324_0001, with dag id: dag_1463520614324_0001_1
2016-05-17 21:30:14,502 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520614324_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:14,503 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520614324_0001_00_000001 given task: attempt_1463520614324_0001_1_00_000000_0
2016-05-17 21:30:14,503 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520614324_0001_1_00_000000_0] started. Is using containerId: [container_1463520614324_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:14,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614324_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CAB878E189D3428A9DA5770B08519658, taskAttemptId=attempt_1463520614324_0001_1_00_000000_0, startTime=1463520614503, containerId=container_1463520614324_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520614324_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:14,503 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520614324_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:14,504 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520614324_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:14,504 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:14,504 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520614324_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520614324_0001_1_00_000000_0
2016-05-17 21:30:14 Starting to run new task attempt: attempt_1463520614324_0001_1_00_000000_0
2016-05-17 21:30:14,505 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520614324_0001_1_00_000000_0
2016-05-17 21:30:14,505 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:14,505 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:14,505 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:14,505 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:14,505 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:14,505 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:14,506 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [92F6A28CBB38435D9E30BCA436212BEA/E707E93543714D2F80A78529824771AD] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL, VertexName: CAB878E189D3428A9DA5770B08519658, VertexParallelism: 1, TaskAttemptID:attempt_1463520614324_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6EFF6FE19A9345D8858C198B870E0A42, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5667E6C4803C4D278A143776703FE96A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:14,506 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:14,508 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520614324_0001_1_00_000000_0
2016-05-17 21:30:14,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:14,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:14,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=6EFF6FE19A9345D8858C198B870E0A42, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:14,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:14,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 6EFF6FE19A9345D8858C198B870E0A42
2016-05-17 21:30:14,513 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 6EFF6FE19A9345D8858C198B870E0A42
2016-05-17 21:30:14,513 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520614324_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:14,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=5667E6C4803C4D278A143776703FE96A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:14,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:14,516 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:14,516 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 6EFF6FE19A9345D8858C198B870E0A42
2016-05-17 21:30:14,516 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 6EFF6FE19A9345D8858C198B870E0A42
2016-05-17 21:30:14,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:14,517 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:14,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 5667E6C4803C4D278A143776703FE96A
2016-05-17 21:30:14,518 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:14,518 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 5667E6C4803C4D278A143776703FE96A
2016-05-17 21:30:14,526 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CAB878E189D3428A9DA5770B08519658, ordinal: 0
2016-05-17 21:30:14,526 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAB878E189D3428A9DA5770B08519658, mem on start (mb), free: 161, total: 364, max: 672
2016-05-17 21:30:14,526 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:14,526 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:14,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:14,531 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:14,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 5667E6C4803C4D278A143776703FE96A
2016-05-17 21:30:14,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:14,532 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:14,533 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:14,534 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:14,534 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:14,535 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:14,535 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:14,535 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:14,536 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 6EFF6FE19A9345D8858C198B870E0A42, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:14,536 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 5667E6C4803C4D278A143776703FE96A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:14,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:14,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 6EFF6FE19A9345D8858C198B870E0A42 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:14,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 6EFF6FE19A9345D8858C198B870E0A42
2016-05-17 21:30:14,538 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:14,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 6EFF6FE19A9345D8858C198B870E0A42
2016-05-17 21:30:14,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:14,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:14,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:14,539 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520614324_0001_1_00_000000_0
2016-05-17 21:30:14,542 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:14,542 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:14,542 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 6EFF6FE19A9345D8858C198B870E0A42
2016-05-17 21:30:14,543 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL"], id: 5667E6C4803C4D278A143776703FE96A
2016-05-17 21:30:14,544 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL"]
2016-05-17 21:30:14,544 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:14,544 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:14,547 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4af0c691
2016-05-17 21:30:14,548 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4af0c691
2016-05-17 21:30:14,549 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:14,549 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:14,550 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CAB878E189D3428A9DA5770B08519658, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:14,553 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:14,554 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:14,554 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520614324_0001_1_00_000000_0
2016-05-17 21:30:14,555 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520614324_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:14,556 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520614324_0001_1_00_000000_0
2016-05-17 21:30:14,556 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAB878E189D3428A9DA5770B08519658, mem on close (mb), free: 159, total: 364, max: 672
2016-05-17 21:30:14 Completed running task attempt: attempt_1463520614324_0001_1_00_000000_0
2016-05-17 21:30:14,558 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520614324_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:14,559 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:14,560 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614324_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CAB878E189D3428A9DA5770B08519658, taskAttemptId=attempt_1463520614324_0001_1_00_000000_0, startTime=1463520614503, finishTime=1463520614559, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520614550, Process_Duration=6, Process_End_Time=1463520614556, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:14,560 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520614324_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:14,560 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:14,560 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520614324_0001_1_00_000000_0
2016-05-17 21:30:14,561 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:14,561 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614324_0001_1][Event:TASK_FINISHED]: vertexName=CAB878E189D3428A9DA5770B08519658, taskId=task_1463520614324_0001_1_00_000000, startTime=1463520614503, finishTime=1463520614561, timeTaken=58, status=SUCCEEDED, successfulAttemptID=attempt_1463520614324_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520614550, Process_Duration=6, Process_End_Time=1463520614556, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:14,562 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520614324_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:14,562 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658] : 1
2016-05-17 21:30:14,562 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658]
2016-05-17 21:30:14,563 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614324_0001_1][Event:VERTEX_FINISHED]: vertexName=CAB878E189D3428A9DA5770B08519658, vertexId=vertex_1463520614324_0001_1_00, initRequestedTime=1463520614481, initedTime=1463520614491, startRequestedTime=1463520614492, startedTime=1463520614492, finishTime=1463520614562, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520614550, Process_Duration=6, Process_End_Time=1463520614556, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520614503, firstTasksToStart=[ task_1463520614324_0001_1_00_000000 ], lastTaskFinishTime=1463520614559, lastTasksToFinish=[ task_1463520614324_0001_1_00_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520614324_0001_1_00_000000 ], longestDurationTasks=[ task_1463520614324_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:14,563 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:14,563 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:14,564 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520614324_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:14,564 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614324_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520614324_0001_00_000001, stoppedTime=1463520614564, exitStatus=0
2016-05-17 21:30:14,564 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520614324_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:14,564 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:14,564 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:14,565 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520614324_0001_1
2016-05-17 21:30:14,565 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614324_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520614324_0001_1
2016-05-17 21:30:14,565 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 5667E6C4803C4D278A143776703FE96A for vertex: vertex_1463520614324_0001_1_00 [CAB878E189D3428A9DA5770B08519658]
2016-05-17 21:30:14,569 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520614550, Process_Duration=6, Process_End_Time=1463520614556, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:14,569 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:14,569 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:14,570 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520614324_0001_00_000001
2016-05-17 21:30:14,570 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520614324_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:14,570 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520614324_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:14,571 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520614324_0001_00_000001, Exiting
2016-05-17 21:30:14,571 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520614324_0001_00_000001
2016-05-17 21:30:14,571 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520614324_0001_00_000001
2016-05-17 21:30:14,574 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520614324_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:14,574 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520614324_0001_00_000001 completed successfully
2016-05-17 21:30:14,577 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520614324_0001_1, queueSize=0
2016-05-17 21:30:14,577 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614324_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520614324_0001_1, startTime=1463520614480, finishTime=1463520614576, timeTaken=96, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520614550, Process_Duration=6, Process_End_Time=1463520614556, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:14,577 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520614324_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:14,578 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520614324_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:14,578 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520614324_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:14 Completed Dag: dag_1463520614324_0001_1
2016-05-17 21:30:14,578 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520614324_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:14,578 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520614324_0001_1_post
2016-05-17 21:30:14,578 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:14,578 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:14,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:14,579 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520614324_0001_00_000001
2016-05-17 21:30:14,585 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ..._RESULTS.ALL_'field2'_ALL, applicationId=application_1463520614324_0001
2016-05-17 21:30:14,587 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:14,587 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:14,587 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:14,587 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:14,587 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:14,588 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:14,588 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:14,588 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:14,588 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:14,588 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:14,589 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:14,589 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:14,589 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:14,594 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/92F6A28CBB38435D9E30BCA436212BEA/E707E93543714D2F80A78529824771AD/.tez/application_1463520614324_0001
2016-05-17 21:30:14,595 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL/_temporary
2016-05-17 21:30:14,595 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.288
2016-05-17 21:30:14,658 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_ALL
2016-05-17 21:30:14,679 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:14,679 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:14,709 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 21:30:14,714 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 21:30:14,714 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:14,715 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 21:30:14,715 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:30:14,729 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:30:14,729 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1
2016-05-17 21:30:14,742 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 21:30:14,742 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:14,742 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1"]
2016-05-17 21:30:14,742 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 21:30:14,742 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 21:30:14,742 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 21:30:14,743 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-17 21:30:14,743 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:30:14,754 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 21:30:14,759 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:14,759 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:30:14,759 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:30:14,768 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/05A77A57F44A41ECA5FD9C0E20E37B76/B7C004809FDB4B8F80B843EF5506FDA7/.tez/application_1463520614759_0001 doesn't exist and is created
2016-05-17 21:30:14,783 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:30:14,783 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:30:14,783 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/05A77A57F44A41ECA5FD9C0E20E37B76/B7C004809FDB4B8F80B843EF5506FDA7/.tez/application_1463520614759_0001
2016-05-17 21:30:14,785 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520614759_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:30:14,794 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:30:14,795 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:30:14,796 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:30:14,796 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:30:14,796 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/05A77A57F44A41ECA5FD9C0E20E37B76/B7C004809FDB4B8F80B843EF5506FDA7/.tez/application_1463520614759_0001/localmode-log-dir
2016-05-17 21:30:14,796 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/05A77A57F44A41ECA5FD9C0E20E37B76/B7C004809FDB4B8F80B843EF5506FDA7/.tez/application_1463520614759_0001/localmode-log-dir/history.txt.appattempt_1463520614759_0001_000000, maxErrors=10
2016-05-17 21:30:14,797 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:30:14,797 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520614759_0001_000000, appSubmitTime=1463520614785, launchTime=1463520614785
2016-05-17 21:30:14,800 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:30:14,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:30:14,803 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58862
2016-05-17 21:30:14,805 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:30:14,805 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:30:14,806 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:30:14,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520614759_0001_000000, startTime=1463520614808
2016-05-17 21:30:14,808 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:30:14,808 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:30:14,883 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:30:14,883 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:30:14,883 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, applicationId=application_1463520614759_0001, dagName=[05A77A57F44A41ECA5FD9C0E20E37B76/B7C004809FDB4B8F80B843EF5506FDA7] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-17 21:30:14,887 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:30:14,888 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:30:14,898 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [05A77A57F44A41ECA5FD9C0E20E37B76/B7C004809FDB4B8F80B843EF5506FDA7] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-17 21:30:14,899 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520614759_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/05A77A57F44A41ECA5FD9C0E20E37B76/B7C004809FDB4B8F80B843EF5506FDA7/.tez/application_1463520614759_0001/localmode-log-dir/dag_1463520614759_0001_1.dot
2016-05-17 21:30:14,900 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/05A77A57F44A41ECA5FD9C0E20E37B76/B7C004809FDB4B8F80B843EF5506FDA7/.tez/application_1463520614759_0001/localmode-log-dir/dag_1463520614759_0001_1-tez-dag.pb.txt
2016-05-17 21:30:14,904 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520614759_0001_1
2016-05-17 21:30:14,904 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:14,904 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [05A77A57F44A41ECA5FD9C0E20E37B76/B7C004809FDB4B8F80B843EF5506FDA7] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-17 21:30:14 Running Dag: dag_1463520614759_0001_1
2016-05-17 21:30:14,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614759_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520614759_0001_1, submitTime=1463520614898
2016-05-17 21:30:14,929 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:30:14,929 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FD3B8F5167A74D69A4A2C9F37C103193
2016-05-17 21:30:14,929 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:30:14,930 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:30:14,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614759_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520614759_0001_1, initTime=1463520614929
2016-05-17 21:30:14,930 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520614759_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:30:14,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614759_0001_1][Event:DAG_STARTED]: dagID=dag_1463520614759_0001_1, startTime=1463520614930
2016-05-17 21:30:14,930 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:30:14,930 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520614759_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:30:14,930 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FD3B8F5167A74D69A4A2C9F37C103193 : {5965B2E209AF43319178DCC15067FF5C={InputName=5965B2E209AF43319178DCC15067FF5C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:30:14,930 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 5965B2E209AF43319178DCC15067FF5C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:30:14,930 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193]
2016-05-17 21:30:14,931 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193]
2016-05-17 21:30:14,931 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193]
2016-05-17 21:30:14,931 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193]. Starting root input initializers: 1
2016-05-17 21:30:14,931 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:30:14,931 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:30:14,932 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, applicationId=application_1463520614759_0001, dagName=[05A77A57F44A41ECA5FD9C0E20E37B76/B7C004809FDB4B8F80B843EF5506FDA7] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-17 21:30:14,933 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 5965B2E209AF43319178DCC15067FF5C on vertex vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193]
2016-05-17 21:30:14,935 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:30:14,935 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 5965B2E209AF43319178DCC15067FF5C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:30:14,935 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:30:14,936 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:30:14,936 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:30:14,937 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 5965B2E209AF43319178DCC15067FF5C on vertex vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193]
2016-05-17 21:30:14,937 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:14,937 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {5965B2E209AF43319178DCC15067FF5C=forAllWorkUnits=true, update=[1]}
2016-05-17 21:30:14,938 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193] parallelism set to 1
2016-05-17 21:30:14,938 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:30:14,938 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614759_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520614759_0001_1_00, eventCount=1
2016-05-17 21:30:14,938 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193]
2016-05-17 21:30:14,938 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9CE93D56EAC6478D9B0E85C6C30AB760, vertexId=vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:30:14,939 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9CE93D56EAC6478D9B0E85C6C30AB760, vertexId=vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193]
2016-05-17 21:30:14,940 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9CE93D56EAC6478D9B0E85C6C30AB760, vertexName=FD3B8F5167A74D69A4A2C9F37C103193, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:30:14,940 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9CE93D56EAC6478D9B0E85C6C30AB760, vertexId=vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193]
2016-05-17 21:30:14,941 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614759_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FD3B8F5167A74D69A4A2C9F37C103193, vertexId=vertex_1463520614759_0001_1_00, initRequestedTime=1463520614930, initedTime=1463520614941, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:30:14,941 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193] with distanceFromRoot: 0
2016-05-17 21:30:14,942 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:30:14,942 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FD3B8F5167A74D69A4A2C9F37C103193
2016-05-17 21:30:14,942 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193]
2016-05-17 21:30:14,942 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:30:14,943 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:30:14,943 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614759_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520614759_0001_1_00, startRequestedTime=1463520614942, startedTime=1463520614942
2016-05-17 21:30:14,943 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:30:14,944 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:30:14,944 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614759_0001_1][Event:TASK_STARTED]: vertexName=FD3B8F5167A74D69A4A2C9F37C103193, taskId=task_1463520614759_0001_1_00_000000, scheduledTime=1463520614943, launchTime=1463520614943
2016-05-17 21:30:14,944 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520614759_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:30:14,944 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193]
2016-05-17 21:30:14,945 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [05A77A57F44A41ECA5FD9C0E20E37B76/B7C004809FDB4B8F80B843EF5506FDA7] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, VertexName: FD3B8F5167A74D69A4A2C9F37C103193, VertexParallelism: 1, TaskAttemptID:attempt_1463520614759_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5965B2E209AF43319178DCC15067FF5C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9CE93D56EAC6478D9B0E85C6C30AB760, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:14,945 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520614759_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:30:14,945 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:30:14,946 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:30:14,946 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463520614759_0001, with dag id: dag_1463520614759_0001_1
2016-05-17 21:30:14,946 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:30:14,947 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:30:14,947 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520614759_0001_00_000001
2016-05-17 21:30:14,947 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520614759_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:30:14,949 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520614759_0001_00_000001
2016-05-17 21:30:14,949 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:30:14,950 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520614759_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:30:14,950 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:30:14,950 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614759_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520614759_0001_00_000001, launchTime=1463520614950
2016-05-17 21:30:14,950 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520614759_0001_00_000001
2016-05-17 21:30:14,952 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520614759_0001_1_00_000000_0] to container: [container_1463520614759_0001_00_000001]
2016-05-17 21:30:14,953 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520614759_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:30:14,953 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520614759_0001_00_000001 given task: attempt_1463520614759_0001_1_00_000000_0
2016-05-17 21:30:14,953 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520614759_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520614759_0001_1_00_000000_0
2016-05-17 21:30:14,953 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520614759_0001_1_00_000000_0] started. Is using containerId: [container_1463520614759_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:30:14,953 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614759_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FD3B8F5167A74D69A4A2C9F37C103193, taskAttemptId=attempt_1463520614759_0001_1_00_000000_0, startTime=1463520614953, containerId=container_1463520614759_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520614759_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:30:14,953 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520614759_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:30:14,953 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520614759_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:30:14,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:30:14 Starting to run new task attempt: attempt_1463520614759_0001_1_00_000000_0
2016-05-17 21:30:14,954 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520614759_0001_1_00_000000_0
2016-05-17 21:30:14,954 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:14,954 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:30:14,954 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:30:14,954 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:30:14,955 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:30:14,955 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:30:14,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [05A77A57F44A41ECA5FD9C0E20E37B76/B7C004809FDB4B8F80B843EF5506FDA7] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, VertexName: FD3B8F5167A74D69A4A2C9F37C103193, VertexParallelism: 1, TaskAttemptID:attempt_1463520614759_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5965B2E209AF43319178DCC15067FF5C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9CE93D56EAC6478D9B0E85C6C30AB760, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:30:14,955 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:30:14,955 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520614759_0001_1_00_000000_0
2016-05-17 21:30:14,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:30:14,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:14,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=5965B2E209AF43319178DCC15067FF5C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:30:14,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:14,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9CE93D56EAC6478D9B0E85C6C30AB760, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:30:14,957 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:30:14,957 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 5965B2E209AF43319178DCC15067FF5C
2016-05-17 21:30:14,957 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 5965B2E209AF43319178DCC15067FF5C
2016-05-17 21:30:14,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:30:14,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9CE93D56EAC6478D9B0E85C6C30AB760
2016-05-17 21:30:14,958 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:30:14,958 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9CE93D56EAC6478D9B0E85C6C30AB760
2016-05-17 21:30:14,959 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:30:14,960 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:30:14,960 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 5965B2E209AF43319178DCC15067FF5C
2016-05-17 21:30:14,960 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 5965B2E209AF43319178DCC15067FF5C
2016-05-17 21:30:14,962 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520614759_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:30:14,968 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FD3B8F5167A74D69A4A2C9F37C103193, ordinal: 0
2016-05-17 21:30:14,968 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FD3B8F5167A74D69A4A2C9F37C103193, mem on start (mb), free: 272, total: 355, max: 672
2016-05-17 21:30:14,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:30:14,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:30:14,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:30:14,974 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:30:14,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9CE93D56EAC6478D9B0E85C6C30AB760
2016-05-17 21:30:14,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:30:14,974 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:30:14,975 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:30:14,975 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:14,975 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:30:14,975 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:30:14,975 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:30:14,975 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 21:30:14,975 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 5965B2E209AF43319178DCC15067FF5C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:30:14,975 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9CE93D56EAC6478D9B0E85C6C30AB760, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:30:14,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:30:14,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 5965B2E209AF43319178DCC15067FF5C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:30:14,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:30:14,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:30:14,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 5965B2E209AF43319178DCC15067FF5C
2016-05-17 21:30:14,976 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:14,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 5965B2E209AF43319178DCC15067FF5C
2016-05-17 21:30:14,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:30:14,976 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520614759_0001_1_00_000000_0
2016-05-17 21:30:14,977 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:30:14,978 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@62c7f808
2016-05-17 21:30:14,979 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@62c7f808
2016-05-17 21:30:14,979 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:30:14,980 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:14,980 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:30:14,980 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 5965B2E209AF43319178DCC15067FF5C
2016-05-17 21:30:14,980 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1"], id: 9CE93D56EAC6478D9B0E85C6C30AB760
2016-05-17 21:30:14,980 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1"]
2016-05-17 21:30:14,981 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:30:14,981 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:30:14,981 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FD3B8F5167A74D69A4A2C9F37C103193, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:30:14,983 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:30:14,984 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:30:14,984 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520614759_0001_1_00_000000_0
2016-05-17 21:30:14,984 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520614759_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 21:30:14,985 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520614759_0001_1_00_000000_0
2016-05-17 21:30:14,985 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FD3B8F5167A74D69A4A2C9F37C103193, mem on close (mb), free: 271, total: 355, max: 672
2016-05-17 21:30:14 Completed running task attempt: attempt_1463520614759_0001_1_00_000000_0
2016-05-17 21:30:14,985 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520614759_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:30:14,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614759_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FD3B8F5167A74D69A4A2C9F37C103193, taskAttemptId=attempt_1463520614759_0001_1_00_000000_0, startTime=1463520614953, finishTime=1463520614988, timeTaken=35, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520614981, Process_Duration=3, Process_End_Time=1463520614984, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:14,988 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520614759_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:30:14,989 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:30:14,989 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520614759_0001_1_00_000000_0
2016-05-17 21:30:14,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614759_0001_1][Event:TASK_FINISHED]: vertexName=FD3B8F5167A74D69A4A2C9F37C103193, taskId=task_1463520614759_0001_1_00_000000, startTime=1463520614953, finishTime=1463520614989, timeTaken=36, status=SUCCEEDED, successfulAttemptID=attempt_1463520614759_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520614981, Process_Duration=3, Process_End_Time=1463520614984, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:14,990 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:30:14,990 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520614759_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:30:14,990 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:30:14,991 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193] : 1
2016-05-17 21:30:14,991 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193]
2016-05-17 21:30:14,991 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:30:14,992 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614759_0001_1][Event:VERTEX_FINISHED]: vertexName=FD3B8F5167A74D69A4A2C9F37C103193, vertexId=vertex_1463520614759_0001_1_00, initRequestedTime=1463520614930, initedTime=1463520614941, startRequestedTime=1463520614942, startedTime=1463520614942, finishTime=1463520614991, timeTaken=49, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520614981, Process_Duration=3, Process_End_Time=1463520614984, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520614953, firstTasksToStart=[ task_1463520614759_0001_1_00_000000 ], lastTaskFinishTime=1463520614988, lastTasksToFinish=[ task_1463520614759_0001_1_00_000000 ], minTaskDuration=35, maxTaskDuration=35, avgTaskDuration=35.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520614759_0001_1_00_000000 ], longestDurationTasks=[ task_1463520614759_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:30:14,992 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:30:14,992 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520614759_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:30:14,993 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614759_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520614759_0001_00_000001, stoppedTime=1463520614993, exitStatus=0
2016-05-17 21:30:14,993 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:30:14,993 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520614759_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:30:14,993 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 21:30:14,993 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520614759_0001_1
2016-05-17 21:30:14,993 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614759_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520614759_0001_1
2016-05-17 21:30:14,994 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 9CE93D56EAC6478D9B0E85C6C30AB760 for vertex: vertex_1463520614759_0001_1_00 [FD3B8F5167A74D69A4A2C9F37C103193]
2016-05-17 21:30:14,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463520614981, Process_Duration=3, Process_End_Time=1463520614984, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 21:30:14,995 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:30:14,995 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:14,995 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520614759_0001_00_000001
2016-05-17 21:30:14,996 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520614759_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:30:14,996 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520614759_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:30:14,996 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520614759_0001_00_000001, Exiting
2016-05-17 21:30:14,996 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520614759_0001_00_000001
2016-05-17 21:30:14,996 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520614759_0001_00_000001
2016-05-17 21:30:14,997 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520614759_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:30:14,997 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520614759_0001_00_000001 completed successfully
2016-05-17 21:30:15,002 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520614759_0001_1, queueSize=0
2016-05-17 21:30:15,002 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:30:15,002 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520614759_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520614759_0001_1, startTime=1463520614930, finishTime=1463520615001, timeTaken=71, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=372768768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463520614981, Process_Duration=3, Process_End_Time=1463520614984, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:30:15,003 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520614759_0001_1 finished with state: SUCCEEDED
2016-05-17 21:30:15,003 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520614759_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:30:15,003 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520614759_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:30:15 Completed Dag: dag_1463520614759_0001_1
2016-05-17 21:30:15,003 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520614759_0001_1, dagState=SUCCEEDED
2016-05-17 21:30:15,003 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520614759_0001_1_post
2016-05-17 21:30:15,003 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:30:15,003 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:30:15,004 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520614759_0001_00_000001
2016-05-17 21:30:15,009 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, applicationId=application_1463520614759_0001
2016-05-17 21:30:15,010 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:30:15,010 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:30:15,010 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:30:15,010 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:30:15,010 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:30:15,011 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:30:15,011 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:30:15,011 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:30:15,012 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:30:15,012 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:30:15,012 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:30:15,013 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:30:15,013 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:30:15,017 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/05A77A57F44A41ECA5FD9C0E20E37B76/B7C004809FDB4B8F80B843EF5506FDA7/.tez/application_1463520614759_0001
2016-05-17 21:30:15,018 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1/_temporary
2016-05-17 21:30:15,018 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.275
2016-05-17 21:30:15,080 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_UNKNOWN_-1
2016-05-17 21:30:15,107 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:30:15,107 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:30:15,145 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-17 21:30:15,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.009
2016-05-17 21:30:15,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:15,147 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:30:15,147 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_UNKNOWN_'field2'