2016-05-17 22:37:31,501 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-17 22:37:31,529 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.detail.EveryEveryPipeAssemblyPlatformTest, with platform: hadoop2-tez
2016-05-17 22:37:32,054 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-17 22:37:32,418 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-17 22:37:32,739 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides:
2016-05-17 22:37:32,902 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:32,903 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:32,992 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: B4C44578366A4CE496A0D541CE99D390
2016-05-17 22:37:33,210 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.086
2016-05-17 22:37:33,221 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.098
2016-05-17 22:37:33,222 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:33,222 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:33,223 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:33,350 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:33,352 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1
2016-05-17 22:37:33,548 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-17 22:37:33,551 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:33,552 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:33,553 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1"]
2016-05-17 22:37:33,553 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:33,554 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:33,554 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:33,555 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-17 22:37:33,556 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:33,576 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:33,728 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:33,734 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:33,735 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:33,776 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7A64ACB810E94CD88AEA7B84D1AECD9F/C0FF8F447EDB4F5084B47EB531777E72/.tez/application_1463524653734_0001 doesn't exist and is created
2016-05-17 22:37:33,840 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:33,840 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:33,841 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7A64ACB810E94CD88AEA7B84D1AECD9F/C0FF8F447EDB4F5084B47EB531777E72/.tez/application_1463524653734_0001
2016-05-17 22:37:33,873 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524653734_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:33,913 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:33,928 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:33,940 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-17 22:37:33,940 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-17 22:37:33,941 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:33,953 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:33,957 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:33,961 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7A64ACB810E94CD88AEA7B84D1AECD9F/C0FF8F447EDB4F5084B47EB531777E72/.tez/application_1463524653734_0001/localmode-log-dir
2016-05-17 22:37:33,962 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7A64ACB810E94CD88AEA7B84D1AECD9F/C0FF8F447EDB4F5084B47EB531777E72/.tez/application_1463524653734_0001/localmode-log-dir/history.txt.appattempt_1463524653734_0001_000000, maxErrors=10
2016-05-17 22:37:33,962 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:33,965 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524653734_0001_000000, appSubmitTime=1463524653858, launchTime=1463524653871
2016-05-17 22:37:33,973 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:33,973 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:33,975 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:33,975 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:34,028 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34838
2016-05-17 22:37:34,041 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 22:37:34,041 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 22:37:34,041 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:34,069 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:34,138 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:34,138 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524653734_0001_000000, startTime=1463524654137
2016-05-17 22:37:34,138 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:34,142 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:34,161 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:34,161 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_'field2'_-1, applicationId=application_1463524653734_0001, dagName=[7A64ACB810E94CD88AEA7B84D1AECD9F/C0FF8F447EDB4F5084B47EB531777E72] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-17 22:37:34,394 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7A64ACB810E94CD88AEA7B84D1AECD9F/C0FF8F447EDB4F5084B47EB531777E72] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-17 22:37:34,491 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524653734_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7A64ACB810E94CD88AEA7B84D1AECD9F/C0FF8F447EDB4F5084B47EB531777E72/.tez/application_1463524653734_0001/localmode-log-dir/dag_1463524653734_0001_1.dot
2016-05-17 22:37:34,494 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7A64ACB810E94CD88AEA7B84D1AECD9F/C0FF8F447EDB4F5084B47EB531777E72/.tez/application_1463524653734_0001/localmode-log-dir/dag_1463524653734_0001_1-tez-dag.pb.txt
2016-05-17 22:37:34,592 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524653734_0001_1
2016-05-17 22:37:34,592 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:34,593 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7A64ACB810E94CD88AEA7B84D1AECD9F/C0FF8F447EDB4F5084B47EB531777E72] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-17 22:37:34 Running Dag: dag_1463524653734_0001_1
2016-05-17 22:37:34,628 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:34,628 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524653734_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524653734_0001_1, submitTime=1463524654395
2016-05-17 22:37:34,673 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C3B8590F66EF4AAE93E02D0C9153170E
2016-05-17 22:37:34,676 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:34,679 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:34,680 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524653734_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524653734_0001_1, initTime=1463524654631
2016-05-17 22:37:34,681 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524653734_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:34,684 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524653734_0001_1][Event:DAG_STARTED]: dagID=dag_1463524653734_0001_1, startTime=1463524654683
2016-05-17 22:37:34,684 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:34,686 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524653734_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:34,686 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:34,686 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C3B8590F66EF4AAE93E02D0C9153170E : {B23F12560DA645C8BB7525D3D8FF1DE2={InputName=B23F12560DA645C8BB7525D3D8FF1DE2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:34,686 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B23F12560DA645C8BB7525D3D8FF1DE2, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:34,686 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E]
2016-05-17 22:37:34,697 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E]
2016-05-17 22:37:34,697 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E]
2016-05-17 22:37:34,699 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_'field2'_-1, applicationId=application_1463524653734_0001, dagName=[7A64ACB810E94CD88AEA7B84D1AECD9F/C0FF8F447EDB4F5084B47EB531777E72] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1
2016-05-17 22:37:34,699 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E]. Starting root input initializers: 1
2016-05-17 22:37:34,716 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B23F12560DA645C8BB7525D3D8FF1DE2 on vertex vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E]
2016-05-17 22:37:34,721 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:34,735 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:34,737 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B23F12560DA645C8BB7525D3D8FF1DE2 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:34,738 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:34,738 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:34,747 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524653734_0001, with dag id: dag_1463524653734_0001_1
2016-05-17 22:37:34,785 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:34,788 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B23F12560DA645C8BB7525D3D8FF1DE2 on vertex vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E]
2016-05-17 22:37:34,798 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:34,798 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B23F12560DA645C8BB7525D3D8FF1DE2=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:34,820 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E] parallelism set to 1
2016-05-17 22:37:34,820 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:34,834 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524653734_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524653734_0001_1_00, eventCount=1
2016-05-17 22:37:34,838 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E]
2016-05-17 22:37:34,838 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=723C616DE4D54304B5EAB53AB08E7469, vertexId=vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:34,842 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=723C616DE4D54304B5EAB53AB08E7469, vertexId=vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E]
2016-05-17 22:37:34,863 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=723C616DE4D54304B5EAB53AB08E7469, vertexName=C3B8590F66EF4AAE93E02D0C9153170E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:34,875 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=723C616DE4D54304B5EAB53AB08E7469, vertexId=vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E]
2016-05-17 22:37:34,880 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:34,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524653734_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C3B8590F66EF4AAE93E02D0C9153170E, vertexId=vertex_1463524653734_0001_1_00, initRequestedTime=1463524654686, initedTime=1463524654878, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:34,881 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E] with distanceFromRoot: 0
2016-05-17 22:37:34,882 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:34,884 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C3B8590F66EF4AAE93E02D0C9153170E
2016-05-17 22:37:34,885 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E]
2016-05-17 22:37:34,887 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:34,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524653734_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524653734_0001_1_00, startRequestedTime=1463524654884, startedTime=1463524654884
2016-05-17 22:37:34,889 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:34,921 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:34,922 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524653734_0001_1][Event:TASK_STARTED]: vertexName=C3B8590F66EF4AAE93E02D0C9153170E, taskId=task_1463524653734_0001_1_00_000000, scheduledTime=1463524654920, launchTime=1463524654920
2016-05-17 22:37:34,922 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524653734_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:34,926 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E]
2016-05-17 22:37:34,929 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7A64ACB810E94CD88AEA7B84D1AECD9F/C0FF8F447EDB4F5084B47EB531777E72] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1, VertexName: C3B8590F66EF4AAE93E02D0C9153170E, VertexParallelism: 1, TaskAttemptID:attempt_1463524653734_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B23F12560DA645C8BB7525D3D8FF1DE2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=723C616DE4D54304B5EAB53AB08E7469, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:34,932 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524653734_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:34,932 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:34,977 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:34,980 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:34,980 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:34,985 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524653734_0001_00_000001
2016-05-17 22:37:34,985 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524653734_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:34,995 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524653734_0001_00_000001
2016-05-17 22:37:34,996 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:35,000 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524653734_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:35,001 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524653734_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524653734_0001_00_000001, launchTime=1463524655001
2016-05-17 22:37:35,001 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:35,005 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524653734_0001_00_000001
2016-05-17 22:37:35,008 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524653734_0001_1_00_000000_0] to container: [container_1463524653734_0001_00_000001]
2016-05-17 22:37:35,009 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524653734_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:35,009 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524653734_0001_00_000001 given task: attempt_1463524653734_0001_1_00_000000_0
2016-05-17 22:37:35,010 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524653734_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524653734_0001_1_00_000000_0
2016-05-17 22:37:35 Starting to run new task attempt: attempt_1463524653734_0001_1_00_000000_0
2016-05-17 22:37:35,011 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524653734_0001_1_00_000000_0
2016-05-17 22:37:35,011 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:35,011 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524653734_0001_1_00_000000_0] started. Is using containerId: [container_1463524653734_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:35,013 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:35,014 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:35,015 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:35,016 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:35,016 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524653734_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C3B8590F66EF4AAE93E02D0C9153170E, taskAttemptId=attempt_1463524653734_0001_1_00_000000_0, startTime=1463524655010, containerId=container_1463524653734_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524653734_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:35,017 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524653734_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:35,018 INFO counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters: GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-17 22:37:35,019 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:35,021 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524653734_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:35,039 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:35,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7A64ACB810E94CD88AEA7B84D1AECD9F/C0FF8F447EDB4F5084B47EB531777E72] every.every/(1/1) ...d'_RESULTS.-1_'field2'_-1, VertexName: C3B8590F66EF4AAE93E02D0C9153170E, VertexParallelism: 1, TaskAttemptID:attempt_1463524653734_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B23F12560DA645C8BB7525D3D8FF1DE2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=723C616DE4D54304B5EAB53AB08E7469, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:35,042 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:35,047 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524653734_0001_1_00_000000_0
2016-05-17 22:37:35,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:35,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:35,051 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524653734_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:35,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B23F12560DA645C8BB7525D3D8FF1DE2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:35,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:35,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:35,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B23F12560DA645C8BB7525D3D8FF1DE2
2016-05-17 22:37:35,063 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B23F12560DA645C8BB7525D3D8FF1DE2
2016-05-17 22:37:35,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=723C616DE4D54304B5EAB53AB08E7469, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:35,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:35,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 723C616DE4D54304B5EAB53AB08E7469
2016-05-17 22:37:35,070 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:35,070 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 723C616DE4D54304B5EAB53AB08E7469
2016-05-17 22:37:35,085 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:35,087 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:35,089 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B23F12560DA645C8BB7525D3D8FF1DE2
2016-05-17 22:37:35,090 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B23F12560DA645C8BB7525D3D8FF1DE2
2016-05-17 22:37:35,110 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:35,110 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 723C616DE4D54304B5EAB53AB08E7469
2016-05-17 22:37:35,113 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C3B8590F66EF4AAE93E02D0C9153170E, ordinal: 0
2016-05-17 22:37:35,114 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C3B8590F66EF4AAE93E02D0C9153170E, mem on start (mb), free: 213, total: 281, max: 672
2016-05-17 22:37:35,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:35,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:35,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:35,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:35,117 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:35,125 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:35,126 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:35,126 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:35,126 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:35,127 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:35,128 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:35,128 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B23F12560DA645C8BB7525D3D8FF1DE2, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:35,128 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 723C616DE4D54304B5EAB53AB08E7469, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:35,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:35,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B23F12560DA645C8BB7525D3D8FF1DE2 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:35,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B23F12560DA645C8BB7525D3D8FF1DE2
2016-05-17 22:37:35,131 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:35,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B23F12560DA645C8BB7525D3D8FF1DE2
2016-05-17 22:37:35,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:35,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:35,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:35,134 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524653734_0001_1_00_000000_0
2016-05-17 22:37:35,135 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:35,144 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7e8958a
2016-05-17 22:37:35,153 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7e8958a
2016-05-17 22:37:35,153 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:35,153 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:35,169 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:35,193 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B23F12560DA645C8BB7525D3D8FF1DE2
2016-05-17 22:37:35,193 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1"], id: 723C616DE4D54304B5EAB53AB08E7469
2016-05-17 22:37:35,194 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1"]
2016-05-17 22:37:35,196 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:35,196 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:35,198 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C3B8590F66EF4AAE93E02D0C9153170E, all 1 inputs ready in: 00:00:00.001
2016-05-17 22:37:35,207 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:35,208 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:35,209 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524653734_0001_1_00_000000_0
2016-05-17 22:37:35,210 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524653734_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:35,212 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524653734_0001_1_00_000000_0
2016-05-17 22:37:35,212 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C3B8590F66EF4AAE93E02D0C9153170E, mem on close (mb), free: 206, total: 281, max: 672
2016-05-17 22:37:35 Completed running task attempt: attempt_1463524653734_0001_1_00_000000_0
2016-05-17 22:37:35,219 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524653734_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:35,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524655198, Process_Duration=13, Process_End_Time=1463524655211, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:35,224 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:35,225 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:35,228 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:35,229 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524653734_0001_00_000001
2016-05-17 22:37:35,232 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524653734_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C3B8590F66EF4AAE93E02D0C9153170E, taskAttemptId=attempt_1463524653734_0001_1_00_000000_0, startTime=1463524655010, finishTime=1463524655221, timeTaken=211, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524655198, Process_Duration=13, Process_End_Time=1463524655211, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:35,233 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524653734_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:35,233 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:35,235 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524653734_0001_1_00_000000_0
2016-05-17 22:37:35,237 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:35,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524653734_0001_1][Event:TASK_FINISHED]: vertexName=C3B8590F66EF4AAE93E02D0C9153170E, taskId=task_1463524653734_0001_1_00_000000, startTime=1463524655010, finishTime=1463524655236, timeTaken=226, status=SUCCEEDED, successfulAttemptID=attempt_1463524653734_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524655198, Process_Duration=13, Process_End_Time=1463524655211, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:35,243 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524653734_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:35,243 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E] : 1
2016-05-17 22:37:35,243 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E]
2016-05-17 22:37:35,250 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:35,254 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524653734_0001_1][Event:VERTEX_FINISHED]: vertexName=C3B8590F66EF4AAE93E02D0C9153170E, vertexId=vertex_1463524653734_0001_1_00, initRequestedTime=1463524654686, initedTime=1463524654878, startRequestedTime=1463524654884, startedTime=1463524654884, finishTime=1463524655243, timeTaken=359, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524655198, Process_Duration=13, Process_End_Time=1463524655211, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524655010, firstTasksToStart=[ task_1463524653734_0001_1_00_000000 ], lastTaskFinishTime=1463524655221, lastTasksToFinish=[ task_1463524653734_0001_1_00_000000 ], minTaskDuration=211, maxTaskDuration=211, avgTaskDuration=211.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524653734_0001_1_00_000000 ], longestDurationTasks=[ task_1463524653734_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:35,255 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:35,256 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524653734_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:35,257 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:35,258 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524653734_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524653734_0001_00_000001, stoppedTime=1463524655257, exitStatus=0
2016-05-17 22:37:35,259 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524653734_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:35,259 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:35,260 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524653734_0001_1
2016-05-17 22:37:35,261 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524653734_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524653734_0001_1
2016-05-17 22:37:35,262 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 723C616DE4D54304B5EAB53AB08E7469 for vertex: vertex_1463524653734_0001_1_00 [C3B8590F66EF4AAE93E02D0C9153170E]
2016-05-17 22:37:35,262 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524653734_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:35,262 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524653734_0001_00_000001: 30 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:35,263 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524653734_0001_00_000001, Exiting
2016-05-17 22:37:35,263 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524653734_0001_00_000001
2016-05-17 22:37:35,263 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524653734_0001_00_000001
2016-05-17 22:37:35,267 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524653734_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:35,267 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524653734_0001_00_000001 completed successfully
2016-05-17 22:37:35,286 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524653734_0001_1, queueSize=0
2016-05-17 22:37:35,287 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:35,288 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524653734_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524653734_0001_1, startTime=1463524654683, finishTime=1463524655280, timeTaken=597, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524655198, Process_Duration=13, Process_End_Time=1463524655211, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:35,290 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524653734_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:35,291 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524653734_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:35,291 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524653734_0001_00_000001
2016-05-17 22:37:35,292 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524653734_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:35 Completed Dag: dag_1463524653734_0001_1
2016-05-17 22:37:35,294 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524653734_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:35,294 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524653734_0001_1_post
2016-05-17 22:37:35,294 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:35,294 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:35,305 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.-1_'field2'_-1, applicationId=application_1463524653734_0001
2016-05-17 22:37:35,309 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:35,310 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:35,310 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:35,310 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:35,310 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:35,311 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:35,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:35,313 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:35,314 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:35,314 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:35,314 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:35,315 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:35,316 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:35,320 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7A64ACB810E94CD88AEA7B84D1AECD9F/C0FF8F447EDB4F5084B47EB531777E72/.tez/application_1463524653734_0001
2016-05-17 22:37:35,322 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_-1/_temporary
2016-05-17 22:37:35,322 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:01.770
2016-05-17 22:37:35,484 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_-1
2016-05-17 22:37:35,530 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:35,531 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:35,610 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.031
2016-05-17 22:37:35,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.046
2016-05-17 22:37:35,628 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:35,628 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:35,628 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:35,654 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:35,655 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'
2016-05-17 22:37:35,691 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:35,692 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:35,692 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'"]
2016-05-17 22:37:35,693 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:35,694 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:35,694 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:35,695 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ULTS.-1_'field2'_'field2'
2016-05-17 22:37:35,696 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:35,720 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:35,726 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:35,727 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:35,727 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:35,738 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/881726EE76EA4BD0AF1F12F555F00289/B33B962E7E7C49F89C9D1FAC35D0B0D1/.tez/application_1463524655726_0001 doesn't exist and is created
2016-05-17 22:37:35,770 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:35,770 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:35,771 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/881726EE76EA4BD0AF1F12F555F00289/B33B962E7E7C49F89C9D1FAC35D0B0D1/.tez/application_1463524655726_0001
2016-05-17 22:37:35,776 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524655726_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:35,831 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:35,833 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:35,837 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:35,837 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:35,838 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/881726EE76EA4BD0AF1F12F555F00289/B33B962E7E7C49F89C9D1FAC35D0B0D1/.tez/application_1463524655726_0001/localmode-log-dir
2016-05-17 22:37:35,839 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/881726EE76EA4BD0AF1F12F555F00289/B33B962E7E7C49F89C9D1FAC35D0B0D1/.tez/application_1463524655726_0001/localmode-log-dir/history.txt.appattempt_1463524655726_0001_000000, maxErrors=10
2016-05-17 22:37:35,839 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:35,840 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524655726_0001_000000, appSubmitTime=1463524655775, launchTime=1463524655776
2016-05-17 22:37:35,849 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:35,853 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:35,858 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55653
2016-05-17 22:37:35,859 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:35,861 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:35,865 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:35,868 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:35,868 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524655726_0001_000000, startTime=1463524655867
2016-05-17 22:37:35,868 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:35,871 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:35,871 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:35,871 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ULTS.-1_'field2'_'field2', applicationId=application_1463524655726_0001, dagName=[881726EE76EA4BD0AF1F12F555F00289/B33B962E7E7C49F89C9D1FAC35D0B0D1] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-17 22:37:35,900 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [881726EE76EA4BD0AF1F12F555F00289/B33B962E7E7C49F89C9D1FAC35D0B0D1] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-17 22:37:35,903 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524655726_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/881726EE76EA4BD0AF1F12F555F00289/B33B962E7E7C49F89C9D1FAC35D0B0D1/.tez/application_1463524655726_0001/localmode-log-dir/dag_1463524655726_0001_1.dot
2016-05-17 22:37:35,905 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/881726EE76EA4BD0AF1F12F555F00289/B33B962E7E7C49F89C9D1FAC35D0B0D1/.tez/application_1463524655726_0001/localmode-log-dir/dag_1463524655726_0001_1-tez-dag.pb.txt
2016-05-17 22:37:35,932 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524655726_0001_1
2016-05-17 22:37:35,933 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:35,933 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [881726EE76EA4BD0AF1F12F555F00289/B33B962E7E7C49F89C9D1FAC35D0B0D1] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-17 22:37:35 Running Dag: dag_1463524655726_0001_1
2016-05-17 22:37:35,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524655726_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524655726_0001_1, submitTime=1463524655900
2016-05-17 22:37:35,949 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:35,950 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A1743ED4CCAF42FC82547599E4DB5C2D
2016-05-17 22:37:35,950 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:35,952 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524655726_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524655726_0001_1, initTime=1463524655949
2016-05-17 22:37:35,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:35,953 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524655726_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:35,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524655726_0001_1][Event:DAG_STARTED]: dagID=dag_1463524655726_0001_1, startTime=1463524655954
2016-05-17 22:37:35,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:35,954 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:35,955 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524655726_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:35,956 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A1743ED4CCAF42FC82547599E4DB5C2D : {59D4E5FB5EA34D5C878ABB04C69CF447={InputName=59D4E5FB5EA34D5C878ABB04C69CF447}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:35,956 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ULTS.-1_'field2'_'field2', applicationId=application_1463524655726_0001, dagName=[881726EE76EA4BD0AF1F12F555F00289/B33B962E7E7C49F89C9D1FAC35D0B0D1] every.every/(1/1) ...ULTS.-1_'field2'_'field2'
2016-05-17 22:37:35,956 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 59D4E5FB5EA34D5C878ABB04C69CF447, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:35,957 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D]
2016-05-17 22:37:35,957 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D]
2016-05-17 22:37:35,957 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D]
2016-05-17 22:37:35,957 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D]. Starting root input initializers: 1
2016-05-17 22:37:35,958 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:35,959 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 59D4E5FB5EA34D5C878ABB04C69CF447 on vertex vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D]
2016-05-17 22:37:35,965 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:35,966 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 59D4E5FB5EA34D5C878ABB04C69CF447 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:35,968 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:35,969 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:35,972 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:35,972 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 59D4E5FB5EA34D5C878ABB04C69CF447 on vertex vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D]
2016-05-17 22:37:35,973 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:35,973 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {59D4E5FB5EA34D5C878ABB04C69CF447=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:35,974 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D] parallelism set to 1
2016-05-17 22:37:35,974 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:35,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524655726_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524655726_0001_1_00, eventCount=1
2016-05-17 22:37:35,975 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D]
2016-05-17 22:37:35,976 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=08655F0469B04F3A8F849CDD4DDF1A3B, vertexId=vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:35,976 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=08655F0469B04F3A8F849CDD4DDF1A3B, vertexId=vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D]
2016-05-17 22:37:35,983 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=08655F0469B04F3A8F849CDD4DDF1A3B, vertexName=A1743ED4CCAF42FC82547599E4DB5C2D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:35,985 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=08655F0469B04F3A8F849CDD4DDF1A3B, vertexId=vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D]
2016-05-17 22:37:35,988 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:35,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524655726_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A1743ED4CCAF42FC82547599E4DB5C2D, vertexId=vertex_1463524655726_0001_1_00, initRequestedTime=1463524655955, initedTime=1463524655987, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:35,989 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D] with distanceFromRoot: 0
2016-05-17 22:37:35,989 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:35,993 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524655726_0001, with dag id: dag_1463524655726_0001_1
2016-05-17 22:37:35,990 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A1743ED4CCAF42FC82547599E4DB5C2D
2016-05-17 22:37:35,993 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D]
2016-05-17 22:37:35,994 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:35,994 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524655726_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524655726_0001_1_00, startRequestedTime=1463524655990, startedTime=1463524655990
2016-05-17 22:37:35,995 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:35,996 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:35,996 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524655726_0001_1][Event:TASK_STARTED]: vertexName=A1743ED4CCAF42FC82547599E4DB5C2D, taskId=task_1463524655726_0001_1_00_000000, scheduledTime=1463524655995, launchTime=1463524655995
2016-05-17 22:37:35,997 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524655726_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:35,998 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D]
2016-05-17 22:37:36,000 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [881726EE76EA4BD0AF1F12F555F00289/B33B962E7E7C49F89C9D1FAC35D0B0D1] every.every/(1/1) ...ULTS.-1_'field2'_'field2', VertexName: A1743ED4CCAF42FC82547599E4DB5C2D, VertexParallelism: 1, TaskAttemptID:attempt_1463524655726_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=59D4E5FB5EA34D5C878ABB04C69CF447, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=08655F0469B04F3A8F849CDD4DDF1A3B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:36,002 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524655726_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:36,004 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:36,005 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:36,006 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:36,007 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:36,008 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524655726_0001_00_000001
2016-05-17 22:37:36,008 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524655726_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:36,013 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524655726_0001_00_000001
2016-05-17 22:37:36,013 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:36,015 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:36,015 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524655726_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:36,015 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524655726_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524655726_0001_00_000001, launchTime=1463524656014
2016-05-17 22:37:36,016 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524655726_0001_00_000001
2016-05-17 22:37:36,018 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524655726_0001_1_00_000000_0] to container: [container_1463524655726_0001_00_000001]
2016-05-17 22:37:36,018 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524655726_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:36,019 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524655726_0001_1_00_000000_0] started. Is using containerId: [container_1463524655726_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:36,019 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:36,019 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524655726_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A1743ED4CCAF42FC82547599E4DB5C2D, taskAttemptId=attempt_1463524655726_0001_1_00_000000_0, startTime=1463524656019, containerId=container_1463524655726_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524655726_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:36,020 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524655726_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:36,020 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524655726_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:36,020 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524655726_0001_00_000001 given task: attempt_1463524655726_0001_1_00_000000_0
2016-05-17 22:37:36,021 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524655726_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524655726_0001_1_00_000000_0
2016-05-17 22:37:36 Starting to run new task attempt: attempt_1463524655726_0001_1_00_000000_0
2016-05-17 22:37:36,021 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524655726_0001_1_00_000000_0
2016-05-17 22:37:36,021 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:36,022 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:36,022 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:36,022 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:36,022 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:36,022 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:36,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [881726EE76EA4BD0AF1F12F555F00289/B33B962E7E7C49F89C9D1FAC35D0B0D1] every.every/(1/1) ...ULTS.-1_'field2'_'field2', VertexName: A1743ED4CCAF42FC82547599E4DB5C2D, VertexParallelism: 1, TaskAttemptID:attempt_1463524655726_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=59D4E5FB5EA34D5C878ABB04C69CF447, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=08655F0469B04F3A8F849CDD4DDF1A3B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:36,022 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:36,023 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524655726_0001_1_00_000000_0
2016-05-17 22:37:36,028 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524655726_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:36,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:36,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:36,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=59D4E5FB5EA34D5C878ABB04C69CF447, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:36,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:36,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:36,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 59D4E5FB5EA34D5C878ABB04C69CF447
2016-05-17 22:37:36,035 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 59D4E5FB5EA34D5C878ABB04C69CF447
2016-05-17 22:37:36,036 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:36,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=08655F0469B04F3A8F849CDD4DDF1A3B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:36,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:36,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 08655F0469B04F3A8F849CDD4DDF1A3B
2016-05-17 22:37:36,042 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:36,042 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 08655F0469B04F3A8F849CDD4DDF1A3B
2016-05-17 22:37:36,055 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:36,055 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 59D4E5FB5EA34D5C878ABB04C69CF447
2016-05-17 22:37:36,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 59D4E5FB5EA34D5C878ABB04C69CF447
2016-05-17 22:37:36,058 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A1743ED4CCAF42FC82547599E4DB5C2D, ordinal: 0
2016-05-17 22:37:36,059 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1743ED4CCAF42FC82547599E4DB5C2D, mem on start (mb), free: 230, total: 281, max: 672
2016-05-17 22:37:36,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:36,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:36,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:36,069 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:36,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 08655F0469B04F3A8F849CDD4DDF1A3B
2016-05-17 22:37:36,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:36,069 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:36,069 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:36,070 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:36,070 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:36,070 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:36,070 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:36,071 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:36,071 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 59D4E5FB5EA34D5C878ABB04C69CF447, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:36,071 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 08655F0469B04F3A8F849CDD4DDF1A3B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:36,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:36,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 59D4E5FB5EA34D5C878ABB04C69CF447 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:36,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:36,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:36,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 59D4E5FB5EA34D5C878ABB04C69CF447
2016-05-17 22:37:36,071 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:36,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 59D4E5FB5EA34D5C878ABB04C69CF447
2016-05-17 22:37:36,072 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:36,072 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524655726_0001_1_00_000000_0
2016-05-17 22:37:36,075 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:36,077 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@64122ef5
2016-05-17 22:37:36,081 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@64122ef5
2016-05-17 22:37:36,081 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:36,082 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:36,084 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:36,085 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 59D4E5FB5EA34D5C878ABB04C69CF447
2016-05-17 22:37:36,085 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'"], id: 08655F0469B04F3A8F849CDD4DDF1A3B
2016-05-17 22:37:36,086 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'"]
2016-05-17 22:37:36,086 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:36,086 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:36,087 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A1743ED4CCAF42FC82547599E4DB5C2D, all 1 inputs ready in: 00:00:00.001
2016-05-17 22:37:36,090 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:36,090 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:36,091 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524655726_0001_1_00_000000_0
2016-05-17 22:37:36,091 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524655726_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:36,092 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524655726_0001_1_00_000000_0
2016-05-17 22:37:36,093 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1743ED4CCAF42FC82547599E4DB5C2D, mem on close (mb), free: 227, total: 281, max: 672
2016-05-17 22:37:36 Completed running task attempt: attempt_1463524655726_0001_1_00_000000_0
2016-05-17 22:37:36,095 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524655726_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:36,096 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524656087, Process_Duration=5, Process_End_Time=1463524656092, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:36,097 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:36,097 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524655726_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A1743ED4CCAF42FC82547599E4DB5C2D, taskAttemptId=attempt_1463524655726_0001_1_00_000000_0, startTime=1463524656019, finishTime=1463524656096, timeTaken=77, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524656087, Process_Duration=5, Process_End_Time=1463524656092, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:36,098 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524655726_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:36,098 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524655726_0001_1_00_000000_0
2016-05-17 22:37:36,098 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:36,099 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:36,100 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:36,100 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:36,100 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524655726_0001_1][Event:TASK_FINISHED]: vertexName=A1743ED4CCAF42FC82547599E4DB5C2D, taskId=task_1463524655726_0001_1_00_000000, startTime=1463524656019, finishTime=1463524656098, timeTaken=79, status=SUCCEEDED, successfulAttemptID=attempt_1463524655726_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524656087, Process_Duration=5, Process_End_Time=1463524656092, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:36,101 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524655726_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:36,101 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D] : 1
2016-05-17 22:37:36,102 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D]
2016-05-17 22:37:36,103 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:36,104 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524655726_0001_1][Event:VERTEX_FINISHED]: vertexName=A1743ED4CCAF42FC82547599E4DB5C2D, vertexId=vertex_1463524655726_0001_1_00, initRequestedTime=1463524655955, initedTime=1463524655987, startRequestedTime=1463524655990, startedTime=1463524655990, finishTime=1463524656102, timeTaken=112, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524656087, Process_Duration=5, Process_End_Time=1463524656092, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524656019, firstTasksToStart=[ task_1463524655726_0001_1_00_000000 ], lastTaskFinishTime=1463524656096, lastTasksToFinish=[ task_1463524655726_0001_1_00_000000 ], minTaskDuration=77, maxTaskDuration=77, avgTaskDuration=77.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524655726_0001_1_00_000000 ], longestDurationTasks=[ task_1463524655726_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:36,104 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:36,105 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524655726_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:36,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524655726_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524655726_0001_00_000001, stoppedTime=1463524656105, exitStatus=0
2016-05-17 22:37:36,105 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524655726_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:36,105 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:36,106 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524655726_0001_1
2016-05-17 22:37:36,106 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524655726_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524655726_0001_1
2016-05-17 22:37:36,106 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 08655F0469B04F3A8F849CDD4DDF1A3B for vertex: vertex_1463524655726_0001_1_00 [A1743ED4CCAF42FC82547599E4DB5C2D]
2016-05-17 22:37:36,104 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524655726_0001_00_000001
2016-05-17 22:37:36,105 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:36,109 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524655726_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:36,109 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524655726_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:36,110 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524655726_0001_00_000001, Exiting
2016-05-17 22:37:36,110 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524655726_0001_00_000001
2016-05-17 22:37:36,110 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524655726_0001_00_000001
2016-05-17 22:37:36,111 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524655726_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:36,112 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524655726_0001_00_000001 completed successfully
2016-05-17 22:37:36,119 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524655726_0001_1, queueSize=0
2016-05-17 22:37:36,119 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:36,121 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524655726_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524655726_0001_1, startTime=1463524655954, finishTime=1463524656118, timeTaken=164, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524656087, Process_Duration=5, Process_End_Time=1463524656092, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:36,121 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524655726_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:36,122 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524655726_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:36,122 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524655726_0001_00_000001
2016-05-17 22:37:36,124 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524655726_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:36 Completed Dag: dag_1463524655726_0001_1
2016-05-17 22:37:36,124 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524655726_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:36,124 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524655726_0001_1_post
2016-05-17 22:37:36,124 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:36,124 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:36,131 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ULTS.-1_'field2'_'field2', applicationId=application_1463524655726_0001
2016-05-17 22:37:36,134 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:36,134 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:36,134 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:36,134 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:36,134 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:36,136 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:36,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:36,137 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:36,137 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:36,138 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:36,139 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:36,140 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:36,140 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:36,147 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/881726EE76EA4BD0AF1F12F555F00289/B33B962E7E7C49F89C9D1FAC35D0B0D1/.tez/application_1463524655726_0001
2016-05-17 22:37:36,148 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field2'/_temporary
2016-05-17 22:37:36,148 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.456
2016-05-17 22:37:36,237 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_'field2'
2016-05-17 22:37:36,265 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:36,266 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:36,324 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-05-17 22:37:36,341 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.042
2016-05-17 22:37:36,342 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:36,342 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:36,342 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:36,360 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:36,360 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'
2016-05-17 22:37:36,381 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:36,382 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:36,382 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'"]
2016-05-17 22:37:36,384 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:36,384 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:36,384 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:36,386 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...SULTS.-1_'field2'_'field'
2016-05-17 22:37:36,386 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:36,403 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:36,408 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:36,408 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:36,409 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:36,421 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8C5CBB07BC2743FBAB941DB7C9BE173C/AA41E35FF9EE42BB811C849FAA5A5D65/.tez/application_1463524656408_0001 doesn't exist and is created
2016-05-17 22:37:36,443 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:36,443 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:36,444 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8C5CBB07BC2743FBAB941DB7C9BE173C/AA41E35FF9EE42BB811C849FAA5A5D65/.tez/application_1463524656408_0001
2016-05-17 22:37:36,450 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524656408_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:36,464 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:36,466 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:36,468 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:36,468 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:36,469 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8C5CBB07BC2743FBAB941DB7C9BE173C/AA41E35FF9EE42BB811C849FAA5A5D65/.tez/application_1463524656408_0001/localmode-log-dir
2016-05-17 22:37:36,470 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8C5CBB07BC2743FBAB941DB7C9BE173C/AA41E35FF9EE42BB811C849FAA5A5D65/.tez/application_1463524656408_0001/localmode-log-dir/history.txt.appattempt_1463524656408_0001_000000, maxErrors=10
2016-05-17 22:37:36,470 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:36,471 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524656408_0001_000000, appSubmitTime=1463524656450, launchTime=1463524656450
2016-05-17 22:37:36,476 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:36,481 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:36,482 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43344
2016-05-17 22:37:36,483 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:36,483 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:36,485 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:36,487 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:36,487 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524656408_0001_000000, startTime=1463524656487
2016-05-17 22:37:36,487 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:36,544 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:36,544 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:36,544 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_'field', applicationId=application_1463524656408_0001, dagName=[8C5CBB07BC2743FBAB941DB7C9BE173C/AA41E35FF9EE42BB811C849FAA5A5D65] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-17 22:37:36,564 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8C5CBB07BC2743FBAB941DB7C9BE173C/AA41E35FF9EE42BB811C849FAA5A5D65] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-17 22:37:36,566 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524656408_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8C5CBB07BC2743FBAB941DB7C9BE173C/AA41E35FF9EE42BB811C849FAA5A5D65/.tez/application_1463524656408_0001/localmode-log-dir/dag_1463524656408_0001_1.dot
2016-05-17 22:37:36,568 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8C5CBB07BC2743FBAB941DB7C9BE173C/AA41E35FF9EE42BB811C849FAA5A5D65/.tez/application_1463524656408_0001/localmode-log-dir/dag_1463524656408_0001_1-tez-dag.pb.txt
2016-05-17 22:37:36,593 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524656408_0001_1
2016-05-17 22:37:36,593 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:36,593 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8C5CBB07BC2743FBAB941DB7C9BE173C/AA41E35FF9EE42BB811C849FAA5A5D65] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-17 22:37:36 Running Dag: dag_1463524656408_0001_1
2016-05-17 22:37:36,609 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:36,610 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524656408_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524656408_0001_1, submitTime=1463524656564
2016-05-17 22:37:36,611 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C953C3028C12417597DE72C3DA01980F
2016-05-17 22:37:36,611 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:36,611 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:36,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524656408_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524656408_0001_1, initTime=1463524656610
2016-05-17 22:37:36,612 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524656408_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:36,613 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:36,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524656408_0001_1][Event:DAG_STARTED]: dagID=dag_1463524656408_0001_1, startTime=1463524656613
2016-05-17 22:37:36,614 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:36,614 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524656408_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:36,614 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C953C3028C12417597DE72C3DA01980F : {0905A412832745359E6AB2F984AB5E37={InputName=0905A412832745359E6AB2F984AB5E37}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:36,614 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0905A412832745359E6AB2F984AB5E37, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:36,614 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F]
2016-05-17 22:37:36,614 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F]
2016-05-17 22:37:36,615 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F]
2016-05-17 22:37:36,615 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F]. Starting root input initializers: 1
2016-05-17 22:37:36,615 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_'field', applicationId=application_1463524656408_0001, dagName=[8C5CBB07BC2743FBAB941DB7C9BE173C/AA41E35FF9EE42BB811C849FAA5A5D65] every.every/(1/1) ...SULTS.-1_'field2'_'field'
2016-05-17 22:37:36,616 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:36,616 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0905A412832745359E6AB2F984AB5E37 on vertex vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F]
2016-05-17 22:37:36,618 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:36,618 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0905A412832745359E6AB2F984AB5E37 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:36,620 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:36,620 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:36,623 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:36,623 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0905A412832745359E6AB2F984AB5E37 on vertex vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F]
2016-05-17 22:37:36,624 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:36,624 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0905A412832745359E6AB2F984AB5E37=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:36,624 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F] parallelism set to 1
2016-05-17 22:37:36,624 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:36,625 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524656408_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524656408_0001_1_00, eventCount=1
2016-05-17 22:37:36,625 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F]
2016-05-17 22:37:36,625 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=60B0C2DC9CCA4CEFB335D74BE0F56143, vertexId=vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:36,625 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=60B0C2DC9CCA4CEFB335D74BE0F56143, vertexId=vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F]
2016-05-17 22:37:36,628 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=60B0C2DC9CCA4CEFB335D74BE0F56143, vertexName=C953C3028C12417597DE72C3DA01980F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:36,628 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=60B0C2DC9CCA4CEFB335D74BE0F56143, vertexId=vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F]
2016-05-17 22:37:36,630 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:36,631 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524656408_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C953C3028C12417597DE72C3DA01980F, vertexId=vertex_1463524656408_0001_1_00, initRequestedTime=1463524656614, initedTime=1463524656630, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:36,631 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F] with distanceFromRoot: 0
2016-05-17 22:37:36,631 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:36,631 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C953C3028C12417597DE72C3DA01980F
2016-05-17 22:37:36,632 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F]
2016-05-17 22:37:36,632 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:36,633 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524656408_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524656408_0001_1_00, startRequestedTime=1463524656631, startedTime=1463524656631
2016-05-17 22:37:36,633 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:36,634 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:36,634 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524656408_0001_1][Event:TASK_STARTED]: vertexName=C953C3028C12417597DE72C3DA01980F, taskId=task_1463524656408_0001_1_00_000000, scheduledTime=1463524656634, launchTime=1463524656634
2016-05-17 22:37:36,635 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524656408_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:36,635 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F]
2016-05-17 22:37:36,636 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8C5CBB07BC2743FBAB941DB7C9BE173C/AA41E35FF9EE42BB811C849FAA5A5D65] every.every/(1/1) ...SULTS.-1_'field2'_'field', VertexName: C953C3028C12417597DE72C3DA01980F, VertexParallelism: 1, TaskAttemptID:attempt_1463524656408_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0905A412832745359E6AB2F984AB5E37, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=60B0C2DC9CCA4CEFB335D74BE0F56143, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:36,636 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524656408_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:36,636 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:36,638 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:36,640 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:36,640 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:36,642 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524656408_0001_00_000001
2016-05-17 22:37:36,642 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524656408_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:36,645 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524656408_0001_00_000001
2016-05-17 22:37:36,645 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524656408_0001, with dag id: dag_1463524656408_0001_1
2016-05-17 22:37:36,645 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:36,646 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:36,646 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524656408_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524656408_0001_00_000001, launchTime=1463524656646
2016-05-17 22:37:36,647 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524656408_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:36,647 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524656408_0001_00_000001
2016-05-17 22:37:36,649 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524656408_0001_1_00_000000_0] to container: [container_1463524656408_0001_00_000001]
2016-05-17 22:37:36,649 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524656408_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:36,650 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524656408_0001_00_000001 given task: attempt_1463524656408_0001_1_00_000000_0
2016-05-17 22:37:36,650 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524656408_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524656408_0001_1_00_000000_0
2016-05-17 22:37:36,650 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524656408_0001_1_00_000000_0] started. Is using containerId: [container_1463524656408_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:36,651 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:36,651 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524656408_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C953C3028C12417597DE72C3DA01980F, taskAttemptId=attempt_1463524656408_0001_1_00_000000_0, startTime=1463524656650, containerId=container_1463524656408_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524656408_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:36,652 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524656408_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:36,652 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524656408_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:36 Starting to run new task attempt: attempt_1463524656408_0001_1_00_000000_0
2016-05-17 22:37:36,653 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524656408_0001_1_00_000000_0
2016-05-17 22:37:36,653 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:36,653 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:36,654 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:36,654 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:36,654 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:36,654 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:36,655 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8C5CBB07BC2743FBAB941DB7C9BE173C/AA41E35FF9EE42BB811C849FAA5A5D65] every.every/(1/1) ...SULTS.-1_'field2'_'field', VertexName: C953C3028C12417597DE72C3DA01980F, VertexParallelism: 1, TaskAttemptID:attempt_1463524656408_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0905A412832745359E6AB2F984AB5E37, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=60B0C2DC9CCA4CEFB335D74BE0F56143, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:36,655 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:36,656 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524656408_0001_1_00_000000_0
2016-05-17 22:37:36,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:36,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:36,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0905A412832745359E6AB2F984AB5E37, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:36,659 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524656408_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:36,660 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:36,660 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0905A412832745359E6AB2F984AB5E37
2016-05-17 22:37:36,660 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0905A412832745359E6AB2F984AB5E37
2016-05-17 22:37:36,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=60B0C2DC9CCA4CEFB335D74BE0F56143, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:36,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:36,661 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:36,661 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 60B0C2DC9CCA4CEFB335D74BE0F56143
2016-05-17 22:37:36,661 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:36,661 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 60B0C2DC9CCA4CEFB335D74BE0F56143
2016-05-17 22:37:36,664 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:36,674 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:36,674 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0905A412832745359E6AB2F984AB5E37
2016-05-17 22:37:36,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0905A412832745359E6AB2F984AB5E37
2016-05-17 22:37:36,678 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C953C3028C12417597DE72C3DA01980F, ordinal: 0
2016-05-17 22:37:36,678 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C953C3028C12417597DE72C3DA01980F, mem on start (mb), free: 147, total: 281, max: 672
2016-05-17 22:37:36,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:36,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:36,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:36,685 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:36,685 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 60B0C2DC9CCA4CEFB335D74BE0F56143
2016-05-17 22:37:36,686 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:36,686 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:36,686 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:36,686 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:36,686 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:36,686 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:36,687 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:36,687 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:36,687 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0905A412832745359E6AB2F984AB5E37, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:36,687 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 60B0C2DC9CCA4CEFB335D74BE0F56143, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:36,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:36,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0905A412832745359E6AB2F984AB5E37 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:36,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0905A412832745359E6AB2F984AB5E37
2016-05-17 22:37:36,687 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:36,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0905A412832745359E6AB2F984AB5E37
2016-05-17 22:37:36,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:36,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:36,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:36,688 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524656408_0001_1_00_000000_0
2016-05-17 22:37:36,713 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:36,714 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@115c8f0e
2016-05-17 22:37:36,718 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@115c8f0e
2016-05-17 22:37:36,718 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:36,718 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:36,719 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:36,720 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0905A412832745359E6AB2F984AB5E37
2016-05-17 22:37:36,720 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'"], id: 60B0C2DC9CCA4CEFB335D74BE0F56143
2016-05-17 22:37:36,721 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'"]
2016-05-17 22:37:36,721 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:36,722 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:36,722 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C953C3028C12417597DE72C3DA01980F, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:36,727 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:36,728 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:36,731 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524656408_0001_1_00_000000_0
2016-05-17 22:37:36,731 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524656408_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:36,732 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524656408_0001_1_00_000000_0
2016-05-17 22:37:36,732 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C953C3028C12417597DE72C3DA01980F, mem on close (mb), free: 361, total: 387, max: 672
2016-05-17 22:37:36 Completed running task attempt: attempt_1463524656408_0001_1_00_000000_0
2016-05-17 22:37:36,734 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524656408_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:36,736 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=23, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405798912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524656722, Process_Duration=10, Process_End_Time=1463524656732, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:36,736 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:36,736 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:36,737 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:36,737 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524656408_0001_00_000001
2016-05-17 22:37:36,738 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524656408_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C953C3028C12417597DE72C3DA01980F, taskAttemptId=attempt_1463524656408_0001_1_00_000000_0, startTime=1463524656650, finishTime=1463524656736, timeTaken=86, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=23, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405798912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524656722, Process_Duration=10, Process_End_Time=1463524656732, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:36,738 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524656408_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:36,738 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524656408_0001_1_00_000000_0
2016-05-17 22:37:36,739 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524656408_0001_1][Event:TASK_FINISHED]: vertexName=C953C3028C12417597DE72C3DA01980F, taskId=task_1463524656408_0001_1_00_000000, startTime=1463524656650, finishTime=1463524656739, timeTaken=89, status=SUCCEEDED, successfulAttemptID=attempt_1463524656408_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=23, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405798912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524656722, Process_Duration=10, Process_End_Time=1463524656732, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:36,740 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524656408_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:36,740 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F] : 1
2016-05-17 22:37:36,740 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F]
2016-05-17 22:37:36,741 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:36,741 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:36,745 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524656408_0001_1][Event:VERTEX_FINISHED]: vertexName=C953C3028C12417597DE72C3DA01980F, vertexId=vertex_1463524656408_0001_1_00, initRequestedTime=1463524656614, initedTime=1463524656630, startRequestedTime=1463524656631, startedTime=1463524656631, finishTime=1463524656740, timeTaken=109, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=23, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405798912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524656722, Process_Duration=10, Process_End_Time=1463524656732, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524656650, firstTasksToStart=[ task_1463524656408_0001_1_00_000000 ], lastTaskFinishTime=1463524656736, lastTasksToFinish=[ task_1463524656408_0001_1_00_000000 ], minTaskDuration=86, maxTaskDuration=86, avgTaskDuration=86.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524656408_0001_1_00_000000 ], longestDurationTasks=[ task_1463524656408_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:36,745 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:36,745 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:36,746 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524656408_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:36,748 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524656408_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524656408_0001_00_000001, stoppedTime=1463524656748, exitStatus=0
2016-05-17 22:37:36,748 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524656408_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:36,749 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:36,749 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524656408_0001_1
2016-05-17 22:37:36,748 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:36,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524656408_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524656408_0001_1
2016-05-17 22:37:36,750 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 60B0C2DC9CCA4CEFB335D74BE0F56143 for vertex: vertex_1463524656408_0001_1_00 [C953C3028C12417597DE72C3DA01980F]
2016-05-17 22:37:36,752 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524656408_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:36,752 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524656408_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:36,752 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524656408_0001_00_000001, Exiting
2016-05-17 22:37:36,752 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524656408_0001_00_000001
2016-05-17 22:37:36,752 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524656408_0001_00_000001
2016-05-17 22:37:36,756 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524656408_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:36,756 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524656408_0001_00_000001 completed successfully
2016-05-17 22:37:36,765 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524656408_0001_1, queueSize=0
2016-05-17 22:37:36,766 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524656408_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524656408_0001_1, startTime=1463524656613, finishTime=1463524656764, timeTaken=151, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=23, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405798912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524656722, Process_Duration=10, Process_End_Time=1463524656732, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:36,766 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524656408_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:36,766 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524656408_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:36,766 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524656408_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:36,766 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524656408_0001_00_000001
2016-05-17 22:37:36 Completed Dag: dag_1463524656408_0001_1
2016-05-17 22:37:36,766 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:36,767 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524656408_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:36,771 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524656408_0001_1_post
2016-05-17 22:37:36,771 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:36,771 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:36,778 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...SULTS.-1_'field2'_'field', applicationId=application_1463524656408_0001
2016-05-17 22:37:36,781 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:36,781 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:36,782 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:36,782 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:36,782 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:36,782 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:36,783 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:36,783 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:36,783 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:36,784 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:36,784 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:36,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:36,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:36,793 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/8C5CBB07BC2743FBAB941DB7C9BE173C/AA41E35FF9EE42BB811C849FAA5A5D65/.tez/application_1463524656408_0001
2016-05-17 22:37:36,793 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_'field'/_temporary
2016-05-17 22:37:36,794 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.412
2016-05-17 22:37:36,894 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_'field'
2016-05-17 22:37:36,920 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:36,921 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:36,976 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-17 22:37:36,993 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.040
2016-05-17 22:37:36,994 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:36,994 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:36,994 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:37,019 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:37,019 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS
2016-05-17 22:37:37,041 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:37,041 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:37,042 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS"]
2016-05-17 22:37:37,042 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:37,043 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:37,043 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:37,044 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-17 22:37:37,045 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:37,065 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:37,070 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:37,071 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:37,071 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:37,084 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A427FC18CC174EA2AB088E4812B20016/DBE054CBB26C4210800641C2CE829917/.tez/application_1463524657071_0001 doesn't exist and is created
2016-05-17 22:37:37,103 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:37,103 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:37,104 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A427FC18CC174EA2AB088E4812B20016/DBE054CBB26C4210800641C2CE829917/.tez/application_1463524657071_0001
2016-05-17 22:37:37,107 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524657071_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:37,119 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:37,120 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:37,121 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:37,122 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:37,122 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A427FC18CC174EA2AB088E4812B20016/DBE054CBB26C4210800641C2CE829917/.tez/application_1463524657071_0001/localmode-log-dir
2016-05-17 22:37:37,122 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A427FC18CC174EA2AB088E4812B20016/DBE054CBB26C4210800641C2CE829917/.tez/application_1463524657071_0001/localmode-log-dir/history.txt.appattempt_1463524657071_0001_000000, maxErrors=10
2016-05-17 22:37:37,122 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:37,123 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524657071_0001_000000, appSubmitTime=1463524657107, launchTime=1463524657107
2016-05-17 22:37:37,127 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:37,132 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:37,134 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:50094
2016-05-17 22:37:37,135 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:37,135 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:37,137 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:37,139 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524657071_0001_000000, startTime=1463524657139
2016-05-17 22:37:37,140 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:37,139 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:37,204 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:37,204 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:37,204 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_RESULTS, applicationId=application_1463524657071_0001, dagName=[A427FC18CC174EA2AB088E4812B20016/DBE054CBB26C4210800641C2CE829917] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-17 22:37:37,224 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A427FC18CC174EA2AB088E4812B20016/DBE054CBB26C4210800641C2CE829917] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-17 22:37:37,226 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524657071_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A427FC18CC174EA2AB088E4812B20016/DBE054CBB26C4210800641C2CE829917/.tez/application_1463524657071_0001/localmode-log-dir/dag_1463524657071_0001_1.dot
2016-05-17 22:37:37,227 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A427FC18CC174EA2AB088E4812B20016/DBE054CBB26C4210800641C2CE829917/.tez/application_1463524657071_0001/localmode-log-dir/dag_1463524657071_0001_1-tez-dag.pb.txt
2016-05-17 22:37:37,262 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524657071_0001_1
2016-05-17 22:37:37,262 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:37,262 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A427FC18CC174EA2AB088E4812B20016/DBE054CBB26C4210800641C2CE829917] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-17 22:37:37 Running Dag: dag_1463524657071_0001_1
2016-05-17 22:37:37,276 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:37,277 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657071_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524657071_0001_1, submitTime=1463524657224
2016-05-17 22:37:37,277 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex AB4ED8D128A849FA878B774E94DD42BA
2016-05-17 22:37:37,278 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:37,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657071_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524657071_0001_1, initTime=1463524657277
2016-05-17 22:37:37,278 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:37,280 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524657071_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:37,280 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657071_0001_1][Event:DAG_STARTED]: dagID=dag_1463524657071_0001_1, startTime=1463524657280
2016-05-17 22:37:37,280 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:37,280 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524657071_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:37,280 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:37,281 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AB4ED8D128A849FA878B774E94DD42BA : {B5AD31B0CC7D4AD497515C4614615551={InputName=B5AD31B0CC7D4AD497515C4614615551}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:37,281 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B5AD31B0CC7D4AD497515C4614615551, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:37,281 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA]
2016-05-17 22:37:37,281 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA]
2016-05-17 22:37:37,281 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA]
2016-05-17 22:37:37,282 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA]. Starting root input initializers: 1
2016-05-17 22:37:37,282 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:37,282 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...SULTS.-1_'field2'_RESULTS, applicationId=application_1463524657071_0001, dagName=[A427FC18CC174EA2AB088E4812B20016/DBE054CBB26C4210800641C2CE829917] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS
2016-05-17 22:37:37,283 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B5AD31B0CC7D4AD497515C4614615551 on vertex vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA]
2016-05-17 22:37:37,284 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:37,284 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B5AD31B0CC7D4AD497515C4614615551 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:37,286 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:37,286 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:37,289 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:37,290 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B5AD31B0CC7D4AD497515C4614615551 on vertex vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA]
2016-05-17 22:37:37,290 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:37,290 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B5AD31B0CC7D4AD497515C4614615551=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:37,290 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA] parallelism set to 1
2016-05-17 22:37:37,291 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:37,291 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657071_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524657071_0001_1_00, eventCount=1
2016-05-17 22:37:37,291 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA]
2016-05-17 22:37:37,291 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=AA3841AE4A794AB188434B3FE8F6F2D5, vertexId=vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:37,291 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=AA3841AE4A794AB188434B3FE8F6F2D5, vertexId=vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA]
2016-05-17 22:37:37,294 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=AA3841AE4A794AB188434B3FE8F6F2D5, vertexName=AB4ED8D128A849FA878B774E94DD42BA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:37,294 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=AA3841AE4A794AB188434B3FE8F6F2D5, vertexId=vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA]
2016-05-17 22:37:37,296 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:37,296 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657071_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AB4ED8D128A849FA878B774E94DD42BA, vertexId=vertex_1463524657071_0001_1_00, initRequestedTime=1463524657281, initedTime=1463524657295, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:37,296 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA] with distanceFromRoot: 0
2016-05-17 22:37:37,296 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:37,296 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AB4ED8D128A849FA878B774E94DD42BA
2016-05-17 22:37:37,296 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA]
2016-05-17 22:37:37,297 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:37,297 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657071_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524657071_0001_1_00, startRequestedTime=1463524657296, startedTime=1463524657296
2016-05-17 22:37:37,298 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:37,299 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:37,299 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657071_0001_1][Event:TASK_STARTED]: vertexName=AB4ED8D128A849FA878B774E94DD42BA, taskId=task_1463524657071_0001_1_00_000000, scheduledTime=1463524657299, launchTime=1463524657299
2016-05-17 22:37:37,299 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524657071_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:37,299 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA]
2016-05-17 22:37:37,300 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A427FC18CC174EA2AB088E4812B20016/DBE054CBB26C4210800641C2CE829917] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS, VertexName: AB4ED8D128A849FA878B774E94DD42BA, VertexParallelism: 1, TaskAttemptID:attempt_1463524657071_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B5AD31B0CC7D4AD497515C4614615551, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AA3841AE4A794AB188434B3FE8F6F2D5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:37,300 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524657071_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:37,300 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:37,301 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:37,302 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:37,302 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:37,303 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524657071_0001_00_000001
2016-05-17 22:37:37,303 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524657071_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:37,306 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524657071_0001_00_000001
2016-05-17 22:37:37,306 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:37,309 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524657071_0001, with dag id: dag_1463524657071_0001_1
2016-05-17 22:37:37,312 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524657071_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:37,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657071_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524657071_0001_00_000001, launchTime=1463524657311
2016-05-17 22:37:37,313 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:37,314 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524657071_0001_00_000001
2016-05-17 22:37:37,315 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524657071_0001_1_00_000000_0] to container: [container_1463524657071_0001_00_000001]
2016-05-17 22:37:37,315 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524657071_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:37,315 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524657071_0001_00_000001 given task: attempt_1463524657071_0001_1_00_000000_0
2016-05-17 22:37:37,316 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524657071_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524657071_0001_1_00_000000_0
2016-05-17 22:37:37,316 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524657071_0001_1_00_000000_0] started. Is using containerId: [container_1463524657071_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:37 Starting to run new task attempt: attempt_1463524657071_0001_1_00_000000_0
2016-05-17 22:37:37,317 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524657071_0001_1_00_000000_0
2016-05-17 22:37:37,318 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:37,318 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:37,318 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:37,318 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:37,318 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:37,318 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:37,319 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A427FC18CC174EA2AB088E4812B20016/DBE054CBB26C4210800641C2CE829917] every.every/(1/1) ...SULTS.-1_'field2'_RESULTS, VertexName: AB4ED8D128A849FA878B774E94DD42BA, VertexParallelism: 1, TaskAttemptID:attempt_1463524657071_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B5AD31B0CC7D4AD497515C4614615551, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AA3841AE4A794AB188434B3FE8F6F2D5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:37,319 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:37,320 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:37,320 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524657071_0001_1_00_000000_0
2016-05-17 22:37:37,320 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:37,321 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:37,321 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657071_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AB4ED8D128A849FA878B774E94DD42BA, taskAttemptId=attempt_1463524657071_0001_1_00_000000_0, startTime=1463524657316, containerId=container_1463524657071_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524657071_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:37,321 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524657071_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:37,321 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524657071_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:37,322 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524657071_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:37,322 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B5AD31B0CC7D4AD497515C4614615551, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:37,323 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AA3841AE4A794AB188434B3FE8F6F2D5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:37,323 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:37,323 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:37,323 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B5AD31B0CC7D4AD497515C4614615551
2016-05-17 22:37:37,323 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B5AD31B0CC7D4AD497515C4614615551
2016-05-17 22:37:37,324 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:37,324 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AA3841AE4A794AB188434B3FE8F6F2D5
2016-05-17 22:37:37,324 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:37,325 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: AA3841AE4A794AB188434B3FE8F6F2D5
2016-05-17 22:37:37,325 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:37,326 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B5AD31B0CC7D4AD497515C4614615551
2016-05-17 22:37:37,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B5AD31B0CC7D4AD497515C4614615551
2016-05-17 22:37:37,327 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:37,338 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:37,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AA3841AE4A794AB188434B3FE8F6F2D5
2016-05-17 22:37:37,340 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AB4ED8D128A849FA878B774E94DD42BA, ordinal: 0
2016-05-17 22:37:37,340 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AB4ED8D128A849FA878B774E94DD42BA, mem on start (mb), free: 278, total: 387, max: 672
2016-05-17 22:37:37,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:37,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:37,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:37,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:37,340 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:37,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:37,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:37,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:37,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:37,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:37,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:37,342 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B5AD31B0CC7D4AD497515C4614615551, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:37,342 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AA3841AE4A794AB188434B3FE8F6F2D5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:37,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:37,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B5AD31B0CC7D4AD497515C4614615551 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:37,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B5AD31B0CC7D4AD497515C4614615551
2016-05-17 22:37:37,342 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:37,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B5AD31B0CC7D4AD497515C4614615551
2016-05-17 22:37:37,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:37,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:37,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:37,343 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524657071_0001_1_00_000000_0
2016-05-17 22:37:37,343 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:37,344 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@197287ed
2016-05-17 22:37:37,344 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@197287ed
2016-05-17 22:37:37,345 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:37,345 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:37,346 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:37,346 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B5AD31B0CC7D4AD497515C4614615551
2016-05-17 22:37:37,347 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS"], id: AA3841AE4A794AB188434B3FE8F6F2D5
2016-05-17 22:37:37,347 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS"]
2016-05-17 22:37:37,347 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:37,347 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:37,348 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AB4ED8D128A849FA878B774E94DD42BA, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:37,351 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:37,351 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:37,352 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524657071_0001_1_00_000000_0
2016-05-17 22:37:37,352 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524657071_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:37,353 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524657071_0001_1_00_000000_0
2016-05-17 22:37:37,353 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AB4ED8D128A849FA878B774E94DD42BA, mem on close (mb), free: 276, total: 387, max: 672
2016-05-17 22:37:37 Completed running task attempt: attempt_1463524657071_0001_1_00_000000_0
2016-05-17 22:37:37,354 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524657071_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:37,355 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405798912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524657348, Process_Duration=5, Process_End_Time=1463524657353, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:37,355 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:37,356 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:37,355 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:37,356 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657071_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AB4ED8D128A849FA878B774E94DD42BA, taskAttemptId=attempt_1463524657071_0001_1_00_000000_0, startTime=1463524657316, finishTime=1463524657355, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405798912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524657348, Process_Duration=5, Process_End_Time=1463524657353, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:37,357 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524657071_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:37,357 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:37,356 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524657071_0001_00_000001
2016-05-17 22:37:37,357 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524657071_0001_1_00_000000_0
2016-05-17 22:37:37,358 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:37,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657071_0001_1][Event:TASK_FINISHED]: vertexName=AB4ED8D128A849FA878B774E94DD42BA, taskId=task_1463524657071_0001_1_00_000000, startTime=1463524657316, finishTime=1463524657358, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463524657071_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405798912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524657348, Process_Duration=5, Process_End_Time=1463524657353, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:37,359 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524657071_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:37,359 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA] : 1
2016-05-17 22:37:37,359 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA]
2016-05-17 22:37:37,362 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657071_0001_1][Event:VERTEX_FINISHED]: vertexName=AB4ED8D128A849FA878B774E94DD42BA, vertexId=vertex_1463524657071_0001_1_00, initRequestedTime=1463524657281, initedTime=1463524657295, startRequestedTime=1463524657296, startedTime=1463524657296, finishTime=1463524657359, timeTaken=63, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405798912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524657348, Process_Duration=5, Process_End_Time=1463524657353, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524657316, firstTasksToStart=[ task_1463524657071_0001_1_00_000000 ], lastTaskFinishTime=1463524657355, lastTasksToFinish=[ task_1463524657071_0001_1_00_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524657071_0001_1_00_000000 ], longestDurationTasks=[ task_1463524657071_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:37,362 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:37,362 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524657071_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:37,364 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:37,365 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657071_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524657071_0001_00_000001, stoppedTime=1463524657365, exitStatus=0
2016-05-17 22:37:37,365 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524657071_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:37,365 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:37,366 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:37,366 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524657071_0001_1
2016-05-17 22:37:37,366 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657071_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524657071_0001_1
2016-05-17 22:37:37,367 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: AA3841AE4A794AB188434B3FE8F6F2D5 for vertex: vertex_1463524657071_0001_1_00 [AB4ED8D128A849FA878B774E94DD42BA]
2016-05-17 22:37:37,369 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524657071_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:37,369 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524657071_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:37,369 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524657071_0001_00_000001, Exiting
2016-05-17 22:37:37,369 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524657071_0001_00_000001
2016-05-17 22:37:37,369 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524657071_0001_00_000001
2016-05-17 22:37:37,371 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524657071_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:37,371 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524657071_0001_00_000001 completed successfully
2016-05-17 22:37:37,381 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524657071_0001_1, queueSize=0
2016-05-17 22:37:37,381 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:37,381 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657071_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524657071_0001_1, startTime=1463524657280, finishTime=1463524657379, timeTaken=99, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405798912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524657348, Process_Duration=5, Process_End_Time=1463524657353, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:37,381 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524657071_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:37,382 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524657071_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:37,383 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524657071_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:37,383 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524657071_0001_00_000001
2016-05-17 22:37:37 Completed Dag: dag_1463524657071_0001_1
2016-05-17 22:37:37,384 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524657071_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:37,385 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524657071_0001_1_post
2016-05-17 22:37:37,385 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:37,385 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:37,390 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...SULTS.-1_'field2'_RESULTS, applicationId=application_1463524657071_0001
2016-05-17 22:37:37,393 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:37,394 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:37,394 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:37,394 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:37,394 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:37,395 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:37,395 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:37,395 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:37,395 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:37,396 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:37,396 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:37,396 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:37,397 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:37,403 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A427FC18CC174EA2AB088E4812B20016/DBE054CBB26C4210800641C2CE829917/.tez/application_1463524657071_0001
2016-05-17 22:37:37,404 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_RESULTS/_temporary
2016-05-17 22:37:37,404 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.362
2016-05-17 22:37:37,489 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_RESULTS
2016-05-17 22:37:37,515 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:37,515 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:37,566 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-17 22:37:37,576 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.032
2016-05-17 22:37:37,577 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:37,577 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:37,578 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:37,594 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:37,595 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL
2016-05-17 22:37:37,608 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:37,608 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:37,609 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL"]
2016-05-17 22:37:37,609 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:37,609 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:37,609 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:37,610 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-17 22:37:37,610 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:37,624 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:37,628 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:37,630 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:37,630 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:37,640 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/74016B5623FB4209BE58E194461403BC/C33C2C93FB644B8397D31813EBD477DE/.tez/application_1463524657629_0001 doesn't exist and is created
2016-05-17 22:37:37,658 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:37,658 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:37,659 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/74016B5623FB4209BE58E194461403BC/C33C2C93FB644B8397D31813EBD477DE/.tez/application_1463524657629_0001
2016-05-17 22:37:37,664 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524657629_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:37,681 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:37,682 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:37,685 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:37,685 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:37,685 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/74016B5623FB4209BE58E194461403BC/C33C2C93FB644B8397D31813EBD477DE/.tez/application_1463524657629_0001/localmode-log-dir
2016-05-17 22:37:37,686 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/74016B5623FB4209BE58E194461403BC/C33C2C93FB644B8397D31813EBD477DE/.tez/application_1463524657629_0001/localmode-log-dir/history.txt.appattempt_1463524657629_0001_000000, maxErrors=10
2016-05-17 22:37:37,687 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:37,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524657629_0001_000000, appSubmitTime=1463524657663, launchTime=1463524657663
2016-05-17 22:37:37,697 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:37,698 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:37,703 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58565
2016-05-17 22:37:37,704 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:37,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:37,705 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:37,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524657629_0001_000000, startTime=1463524657707
2016-05-17 22:37:37,707 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:37,707 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:37,758 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:37,758 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:37,758 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_'field2'_ALL, applicationId=application_1463524657629_0001, dagName=[74016B5623FB4209BE58E194461403BC/C33C2C93FB644B8397D31813EBD477DE] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-17 22:37:37,786 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [74016B5623FB4209BE58E194461403BC/C33C2C93FB644B8397D31813EBD477DE] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-17 22:37:37,788 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524657629_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/74016B5623FB4209BE58E194461403BC/C33C2C93FB644B8397D31813EBD477DE/.tez/application_1463524657629_0001/localmode-log-dir/dag_1463524657629_0001_1.dot
2016-05-17 22:37:37,789 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/74016B5623FB4209BE58E194461403BC/C33C2C93FB644B8397D31813EBD477DE/.tez/application_1463524657629_0001/localmode-log-dir/dag_1463524657629_0001_1-tez-dag.pb.txt
2016-05-17 22:37:37,809 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524657629_0001_1
2016-05-17 22:37:37,810 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:37,810 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [74016B5623FB4209BE58E194461403BC/C33C2C93FB644B8397D31813EBD477DE] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-17 22:37:37 Running Dag: dag_1463524657629_0001_1
2016-05-17 22:37:37,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657629_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524657629_0001_1, submitTime=1463524657786
2016-05-17 22:37:37,824 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:37,825 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C82AE03D4C604D02AE5A23E914FE429F
2016-05-17 22:37:37,825 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:37,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657629_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524657629_0001_1, initTime=1463524657824
2016-05-17 22:37:37,826 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:37,826 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524657629_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:37,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657629_0001_1][Event:DAG_STARTED]: dagID=dag_1463524657629_0001_1, startTime=1463524657827
2016-05-17 22:37:37,827 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:37,827 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:37,828 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524657629_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:37,828 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C82AE03D4C604D02AE5A23E914FE429F : {95D3387449DA4EC39EE61D547BDA0033={InputName=95D3387449DA4EC39EE61D547BDA0033}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:37,828 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_'field2'_ALL, applicationId=application_1463524657629_0001, dagName=[74016B5623FB4209BE58E194461403BC/C33C2C93FB644B8397D31813EBD477DE] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL
2016-05-17 22:37:37,828 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 95D3387449DA4EC39EE61D547BDA0033, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:37,829 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F]
2016-05-17 22:37:37,829 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F]
2016-05-17 22:37:37,829 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F]
2016-05-17 22:37:37,830 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F]. Starting root input initializers: 1
2016-05-17 22:37:37,831 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:37,831 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 95D3387449DA4EC39EE61D547BDA0033 on vertex vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F]
2016-05-17 22:37:37,834 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:37,834 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 95D3387449DA4EC39EE61D547BDA0033 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:37,836 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:37,836 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:37,839 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:37,840 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 95D3387449DA4EC39EE61D547BDA0033 on vertex vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F]
2016-05-17 22:37:37,840 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:37,841 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {95D3387449DA4EC39EE61D547BDA0033=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:37,841 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F] parallelism set to 1
2016-05-17 22:37:37,841 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:37,841 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657629_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524657629_0001_1_00, eventCount=1
2016-05-17 22:37:37,841 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F]
2016-05-17 22:37:37,841 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=5AE72161425F497993E33B22EC95696D, vertexId=vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:37,842 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=5AE72161425F497993E33B22EC95696D, vertexId=vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F]
2016-05-17 22:37:37,846 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=5AE72161425F497993E33B22EC95696D, vertexName=C82AE03D4C604D02AE5A23E914FE429F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:37,846 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=5AE72161425F497993E33B22EC95696D, vertexId=vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F]
2016-05-17 22:37:37,848 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657629_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C82AE03D4C604D02AE5A23E914FE429F, vertexId=vertex_1463524657629_0001_1_00, initRequestedTime=1463524657828, initedTime=1463524657848, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:37,848 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:37,848 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F] with distanceFromRoot: 0
2016-05-17 22:37:37,849 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:37,849 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C82AE03D4C604D02AE5A23E914FE429F
2016-05-17 22:37:37,849 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F]
2016-05-17 22:37:37,850 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:37,850 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657629_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524657629_0001_1_00, startRequestedTime=1463524657849, startedTime=1463524657849
2016-05-17 22:37:37,851 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:37,851 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:37,852 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657629_0001_1][Event:TASK_STARTED]: vertexName=C82AE03D4C604D02AE5A23E914FE429F, taskId=task_1463524657629_0001_1_00_000000, scheduledTime=1463524657851, launchTime=1463524657851
2016-05-17 22:37:37,852 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524657629_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:37,852 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F]
2016-05-17 22:37:37,853 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [74016B5623FB4209BE58E194461403BC/C33C2C93FB644B8397D31813EBD477DE] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL, VertexName: C82AE03D4C604D02AE5A23E914FE429F, VertexParallelism: 1, TaskAttemptID:attempt_1463524657629_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=95D3387449DA4EC39EE61D547BDA0033, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5AE72161425F497993E33B22EC95696D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:37,853 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524657629_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:37,854 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:37,855 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:37,855 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:37,856 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:37,857 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524657629_0001_00_000001
2016-05-17 22:37:37,857 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524657629_0001, with dag id: dag_1463524657629_0001_1
2016-05-17 22:37:37,857 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524657629_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:37,861 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524657629_0001_00_000001
2016-05-17 22:37:37,862 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:37,863 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524657629_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:37,864 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:37,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657629_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524657629_0001_00_000001, launchTime=1463524657862
2016-05-17 22:37:37,865 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524657629_0001_00_000001
2016-05-17 22:37:37,866 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524657629_0001_1_00_000000_0] to container: [container_1463524657629_0001_00_000001]
2016-05-17 22:37:37,866 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524657629_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:37,866 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524657629_0001_1_00_000000_0] started. Is using containerId: [container_1463524657629_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:37,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:37,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657629_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C82AE03D4C604D02AE5A23E914FE429F, taskAttemptId=attempt_1463524657629_0001_1_00_000000_0, startTime=1463524657866, containerId=container_1463524657629_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524657629_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:37,867 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524657629_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:37,867 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524657629_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:37,867 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524657629_0001_00_000001 given task: attempt_1463524657629_0001_1_00_000000_0
2016-05-17 22:37:37,868 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524657629_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524657629_0001_1_00_000000_0
2016-05-17 22:37:37 Starting to run new task attempt: attempt_1463524657629_0001_1_00_000000_0
2016-05-17 22:37:37,868 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524657629_0001_1_00_000000_0
2016-05-17 22:37:37,868 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:37,868 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:37,868 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:37,868 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:37,869 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:37,869 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:37,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [74016B5623FB4209BE58E194461403BC/C33C2C93FB644B8397D31813EBD477DE] every.every/(1/1) ...'_RESULTS.-1_'field2'_ALL, VertexName: C82AE03D4C604D02AE5A23E914FE429F, VertexParallelism: 1, TaskAttemptID:attempt_1463524657629_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=95D3387449DA4EC39EE61D547BDA0033, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5AE72161425F497993E33B22EC95696D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:37,869 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:37,870 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524657629_0001_1_00_000000_0
2016-05-17 22:37:37,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:37,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:37,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:37,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=5AE72161425F497993E33B22EC95696D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:37,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:37,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 5AE72161425F497993E33B22EC95696D
2016-05-17 22:37:37,875 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:37,875 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 5AE72161425F497993E33B22EC95696D
2016-05-17 22:37:37,875 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524657629_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:37,874 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=95D3387449DA4EC39EE61D547BDA0033, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:37,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:37,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 95D3387449DA4EC39EE61D547BDA0033
2016-05-17 22:37:37,878 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 95D3387449DA4EC39EE61D547BDA0033
2016-05-17 22:37:37,880 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:37,884 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:37,884 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 95D3387449DA4EC39EE61D547BDA0033
2016-05-17 22:37:37,884 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 95D3387449DA4EC39EE61D547BDA0033
2016-05-17 22:37:37,891 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:37,891 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 5AE72161425F497993E33B22EC95696D
2016-05-17 22:37:37,895 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C82AE03D4C604D02AE5A23E914FE429F, ordinal: 0
2016-05-17 22:37:37,895 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C82AE03D4C604D02AE5A23E914FE429F, mem on start (mb), free: 197, total: 387, max: 672
2016-05-17 22:37:37,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:37,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:37,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:37,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:37,895 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:37,896 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:37,896 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:37,896 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:37,896 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:37,896 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:37,897 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:37,897 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 5AE72161425F497993E33B22EC95696D, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:37,897 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 95D3387449DA4EC39EE61D547BDA0033, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:37,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:37,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 95D3387449DA4EC39EE61D547BDA0033 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:37,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:37,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:37,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 95D3387449DA4EC39EE61D547BDA0033
2016-05-17 22:37:37,898 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:37,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 95D3387449DA4EC39EE61D547BDA0033
2016-05-17 22:37:37,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:37,899 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524657629_0001_1_00_000000_0
2016-05-17 22:37:37,899 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:37,900 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@53222823
2016-05-17 22:37:37,901 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@53222823
2016-05-17 22:37:37,901 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:37,901 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:37,905 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:37,906 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 95D3387449DA4EC39EE61D547BDA0033
2016-05-17 22:37:37,906 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL"], id: 5AE72161425F497993E33B22EC95696D
2016-05-17 22:37:37,907 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL"]
2016-05-17 22:37:37,907 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:37,907 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:37,907 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C82AE03D4C604D02AE5A23E914FE429F, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:37,911 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:37,911 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:37,912 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524657629_0001_1_00_000000_0
2016-05-17 22:37:37,912 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524657629_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:37,913 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524657629_0001_1_00_000000_0
2016-05-17 22:37:37,913 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C82AE03D4C604D02AE5A23E914FE429F, mem on close (mb), free: 196, total: 387, max: 672
2016-05-17 22:37:37 Completed running task attempt: attempt_1463524657629_0001_1_00_000000_0
2016-05-17 22:37:37,914 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524657629_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:37,915 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405798912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524657908, Process_Duration=5, Process_End_Time=1463524657913, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:37,915 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:37,916 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:37,915 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:37,918 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524657629_0001_00_000001
2016-05-17 22:37:37,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657629_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C82AE03D4C604D02AE5A23E914FE429F, taskAttemptId=attempt_1463524657629_0001_1_00_000000_0, startTime=1463524657866, finishTime=1463524657915, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405798912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524657908, Process_Duration=5, Process_End_Time=1463524657913, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:37,918 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524657629_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:37,919 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:37,919 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524657629_0001_1_00_000000_0
2016-05-17 22:37:37,920 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:37,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657629_0001_1][Event:TASK_FINISHED]: vertexName=C82AE03D4C604D02AE5A23E914FE429F, taskId=task_1463524657629_0001_1_00_000000, startTime=1463524657866, finishTime=1463524657920, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1463524657629_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405798912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524657908, Process_Duration=5, Process_End_Time=1463524657913, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:37,920 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524657629_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:37,921 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F] : 1
2016-05-17 22:37:37,921 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F]
2016-05-17 22:37:37,922 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:37,923 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657629_0001_1][Event:VERTEX_FINISHED]: vertexName=C82AE03D4C604D02AE5A23E914FE429F, vertexId=vertex_1463524657629_0001_1_00, initRequestedTime=1463524657828, initedTime=1463524657848, startRequestedTime=1463524657849, startedTime=1463524657849, finishTime=1463524657921, timeTaken=72, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405798912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524657908, Process_Duration=5, Process_End_Time=1463524657913, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524657866, firstTasksToStart=[ task_1463524657629_0001_1_00_000000 ], lastTaskFinishTime=1463524657915, lastTasksToFinish=[ task_1463524657629_0001_1_00_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524657629_0001_1_00_000000 ], longestDurationTasks=[ task_1463524657629_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:37,923 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:37,923 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524657629_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:37,923 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657629_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524657629_0001_00_000001, stoppedTime=1463524657923, exitStatus=0
2016-05-17 22:37:37,923 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:37,923 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524657629_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:37,925 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:37,925 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524657629_0001_1
2016-05-17 22:37:37,925 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657629_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524657629_0001_1
2016-05-17 22:37:37,925 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 5AE72161425F497993E33B22EC95696D for vertex: vertex_1463524657629_0001_1_00 [C82AE03D4C604D02AE5A23E914FE429F]
2016-05-17 22:37:37,929 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524657629_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:37,929 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524657629_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:37,929 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524657629_0001_00_000001, Exiting
2016-05-17 22:37:37,930 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524657629_0001_00_000001
2016-05-17 22:37:37,930 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524657629_0001_00_000001
2016-05-17 22:37:37,933 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524657629_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:37,933 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524657629_0001_00_000001 completed successfully
2016-05-17 22:37:37,937 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524657629_0001_1, queueSize=0
2016-05-17 22:37:37,937 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:37,939 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524657629_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524657629_0001_1, startTime=1463524657827, finishTime=1463524657936, timeTaken=109, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405798912, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524657908, Process_Duration=5, Process_End_Time=1463524657913, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:37,939 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524657629_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:37,939 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524657629_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:37,940 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524657629_0001_00_000001
2016-05-17 22:37:37,940 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524657629_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:37 Completed Dag: dag_1463524657629_0001_1
2016-05-17 22:37:37,944 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524657629_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:37,944 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524657629_0001_1_post
2016-05-17 22:37:37,944 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:37,944 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:37,947 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...'_RESULTS.-1_'field2'_ALL, applicationId=application_1463524657629_0001
2016-05-17 22:37:37,951 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:37,951 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:37,951 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:37,951 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:37,951 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:37,952 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:37,952 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:37,952 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:37,953 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:37,953 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:37,953 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:37,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:37,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:37,958 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/74016B5623FB4209BE58E194461403BC/C33C2C93FB644B8397D31813EBD477DE/.tez/application_1463524657629_0001
2016-05-17 22:37:37,959 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_'field2'_ALL/_temporary
2016-05-17 22:37:37,959 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.350
2016-05-17 22:37:38,058 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_'field2'_ALL
2016-05-17 22:37:38,091 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:38,092 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:38,151 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-17 22:37:38,161 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-17 22:37:38,162 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:38,162 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:38,163 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:38,184 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:38,185 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1
2016-05-17 22:37:38,203 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:38,203 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:38,203 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1"]
2016-05-17 22:37:38,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:38,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:38,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:38,205 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-17 22:37:38,205 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:38,249 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:38,254 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:38,255 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:38,255 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:38,264 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ADDAD650261D476C8ACF2B1208BFFC1B/D258FD57E81C4CD8B523F3B05845A76F/.tez/application_1463524658255_0001 doesn't exist and is created
2016-05-17 22:37:38,283 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:38,283 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:38,284 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ADDAD650261D476C8ACF2B1208BFFC1B/D258FD57E81C4CD8B523F3B05845A76F/.tez/application_1463524658255_0001
2016-05-17 22:37:38,287 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524658255_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:38,298 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:38,299 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:38,301 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:38,301 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:38,301 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ADDAD650261D476C8ACF2B1208BFFC1B/D258FD57E81C4CD8B523F3B05845A76F/.tez/application_1463524658255_0001/localmode-log-dir
2016-05-17 22:37:38,302 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ADDAD650261D476C8ACF2B1208BFFC1B/D258FD57E81C4CD8B523F3B05845A76F/.tez/application_1463524658255_0001/localmode-log-dir/history.txt.appattempt_1463524658255_0001_000000, maxErrors=10
2016-05-17 22:37:38,302 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:38,302 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524658255_0001_000000, appSubmitTime=1463524658287, launchTime=1463524658287
2016-05-17 22:37:38,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:38,306 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:38,308 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:38,309 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:38,309 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:33272
2016-05-17 22:37:38,313 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:38,315 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524658255_0001_000000, startTime=1463524658315
2016-05-17 22:37:38,315 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:38,315 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:38,383 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:38,384 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:38,384 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, applicationId=application_1463524658255_0001, dagName=[ADDAD650261D476C8ACF2B1208BFFC1B/D258FD57E81C4CD8B523F3B05845A76F] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-17 22:37:38,402 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [ADDAD650261D476C8ACF2B1208BFFC1B/D258FD57E81C4CD8B523F3B05845A76F] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-17 22:37:38,403 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524658255_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ADDAD650261D476C8ACF2B1208BFFC1B/D258FD57E81C4CD8B523F3B05845A76F/.tez/application_1463524658255_0001/localmode-log-dir/dag_1463524658255_0001_1.dot
2016-05-17 22:37:38,404 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ADDAD650261D476C8ACF2B1208BFFC1B/D258FD57E81C4CD8B523F3B05845A76F/.tez/application_1463524658255_0001/localmode-log-dir/dag_1463524658255_0001_1-tez-dag.pb.txt
2016-05-17 22:37:38,432 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524658255_0001_1
2016-05-17 22:37:38,432 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:38,432 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [ADDAD650261D476C8ACF2B1208BFFC1B/D258FD57E81C4CD8B523F3B05845A76F] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-17 22:37:38 Running Dag: dag_1463524658255_0001_1
2016-05-17 22:37:38,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524658255_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524658255_0001_1, submitTime=1463524658402
2016-05-17 22:37:38,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:38,453 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C89F90374A32436DBFA862B447C09DF1
2016-05-17 22:37:38,454 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:38,454 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524658255_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524658255_0001_1, initTime=1463524658453
2016-05-17 22:37:38,454 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524658255_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:38,455 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:38,456 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524658255_0001_1][Event:DAG_STARTED]: dagID=dag_1463524658255_0001_1, startTime=1463524658456
2016-05-17 22:37:38,456 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:38,457 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, applicationId=application_1463524658255_0001, dagName=[ADDAD650261D476C8ACF2B1208BFFC1B/D258FD57E81C4CD8B523F3B05845A76F] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1
2016-05-17 22:37:38,457 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524658255_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:38,458 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C89F90374A32436DBFA862B447C09DF1 : {A9A8D577C5D64265BAAFF1103F54DBF5={InputName=A9A8D577C5D64265BAAFF1103F54DBF5}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:38,458 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A9A8D577C5D64265BAAFF1103F54DBF5, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:38,458 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1]
2016-05-17 22:37:38,458 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1]
2016-05-17 22:37:38,458 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1]
2016-05-17 22:37:38,458 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1]. Starting root input initializers: 1
2016-05-17 22:37:38,464 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:38,464 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A9A8D577C5D64265BAAFF1103F54DBF5 on vertex vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1]
2016-05-17 22:37:38,464 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:38,468 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:38,469 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A9A8D577C5D64265BAAFF1103F54DBF5 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:38,470 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:38,471 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:38,474 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:38,475 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A9A8D577C5D64265BAAFF1103F54DBF5 on vertex vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1]
2016-05-17 22:37:38,475 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:38,476 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A9A8D577C5D64265BAAFF1103F54DBF5=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:38,476 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1] parallelism set to 1
2016-05-17 22:37:38,476 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:38,476 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524658255_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524658255_0001_1_00, eventCount=1
2016-05-17 22:37:38,477 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524658255_0001, with dag id: dag_1463524658255_0001_1
2016-05-17 22:37:38,477 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1]
2016-05-17 22:37:38,477 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A93506A481F7459883BAC162B887411C, vertexId=vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:38,477 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A93506A481F7459883BAC162B887411C, vertexId=vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1]
2016-05-17 22:37:38,480 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A93506A481F7459883BAC162B887411C, vertexName=C89F90374A32436DBFA862B447C09DF1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:38,480 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A93506A481F7459883BAC162B887411C, vertexId=vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1]
2016-05-17 22:37:38,482 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:38,483 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524658255_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C89F90374A32436DBFA862B447C09DF1, vertexId=vertex_1463524658255_0001_1_00, initRequestedTime=1463524658458, initedTime=1463524658482, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:38,483 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1] with distanceFromRoot: 0
2016-05-17 22:37:38,483 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:38,483 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C89F90374A32436DBFA862B447C09DF1
2016-05-17 22:37:38,483 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1]
2016-05-17 22:37:38,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:38,484 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524658255_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524658255_0001_1_00, startRequestedTime=1463524658483, startedTime=1463524658483
2016-05-17 22:37:38,484 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:38,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524658255_0001_1][Event:TASK_STARTED]: vertexName=C89F90374A32436DBFA862B447C09DF1, taskId=task_1463524658255_0001_1_00_000000, scheduledTime=1463524658484, launchTime=1463524658484
2016-05-17 22:37:38,486 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:38,487 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524658255_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:38,488 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1]
2016-05-17 22:37:38,490 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [ADDAD650261D476C8ACF2B1208BFFC1B/D258FD57E81C4CD8B523F3B05845A76F] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, VertexName: C89F90374A32436DBFA862B447C09DF1, VertexParallelism: 1, TaskAttemptID:attempt_1463524658255_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A9A8D577C5D64265BAAFF1103F54DBF5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A93506A481F7459883BAC162B887411C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:38,491 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524658255_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:38,491 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:38,492 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:38,495 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:38,495 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:38,496 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524658255_0001_00_000001
2016-05-17 22:37:38,496 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524658255_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:38,500 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524658255_0001_00_000001
2016-05-17 22:37:38,500 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:38,502 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524658255_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:38,503 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:38,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524658255_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524658255_0001_00_000001, launchTime=1463524658502
2016-05-17 22:37:38,504 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524658255_0001_00_000001
2016-05-17 22:37:38,506 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524658255_0001_1_00_000000_0] to container: [container_1463524658255_0001_00_000001]
2016-05-17 22:37:38,506 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524658255_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:38,509 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524658255_0001_00_000001 given task: attempt_1463524658255_0001_1_00_000000_0
2016-05-17 22:37:38,509 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524658255_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524658255_0001_1_00_000000_0
2016-05-17 22:37:38,509 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524658255_0001_1_00_000000_0] started. Is using containerId: [container_1463524658255_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:38 Starting to run new task attempt: attempt_1463524658255_0001_1_00_000000_0
2016-05-17 22:37:38,510 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:38,510 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524658255_0001_1_00_000000_0
2016-05-17 22:37:38,510 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524658255_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C89F90374A32436DBFA862B447C09DF1, taskAttemptId=attempt_1463524658255_0001_1_00_000000_0, startTime=1463524658507, containerId=container_1463524658255_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524658255_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:38,510 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524658255_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:38,510 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524658255_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:38,510 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:38,512 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:38,512 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:38,512 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:38,512 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:38,512 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:38,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [ADDAD650261D476C8ACF2B1208BFFC1B/D258FD57E81C4CD8B523F3B05845A76F] every.every/(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, VertexName: C89F90374A32436DBFA862B447C09DF1, VertexParallelism: 1, TaskAttemptID:attempt_1463524658255_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A9A8D577C5D64265BAAFF1103F54DBF5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A93506A481F7459883BAC162B887411C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:38,513 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:38,515 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524658255_0001_1_00_000000_0
2016-05-17 22:37:38,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:38,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:38,519 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A9A8D577C5D64265BAAFF1103F54DBF5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:38,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:38,521 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A9A8D577C5D64265BAAFF1103F54DBF5
2016-05-17 22:37:38,521 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A9A8D577C5D64265BAAFF1103F54DBF5
2016-05-17 22:37:38,523 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:38,525 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524658255_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:38,525 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:38,526 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A9A8D577C5D64265BAAFF1103F54DBF5
2016-05-17 22:37:38,526 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A9A8D577C5D64265BAAFF1103F54DBF5
2016-05-17 22:37:38,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A93506A481F7459883BAC162B887411C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:38,528 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:38,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:38,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A93506A481F7459883BAC162B887411C
2016-05-17 22:37:38,532 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:38,533 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A93506A481F7459883BAC162B887411C
2016-05-17 22:37:38,544 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C89F90374A32436DBFA862B447C09DF1, ordinal: 0
2016-05-17 22:37:38,544 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C89F90374A32436DBFA862B447C09DF1, mem on start (mb), free: 322, total: 383, max: 672
2016-05-17 22:37:38,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:38,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:38,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:38,548 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:38,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A93506A481F7459883BAC162B887411C
2016-05-17 22:37:38,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:38,549 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:38,550 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:38,550 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:38,550 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:38,551 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:38,552 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:38,552 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:38,552 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A9A8D577C5D64265BAAFF1103F54DBF5, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:38,553 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A93506A481F7459883BAC162B887411C, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:38,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:38,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A9A8D577C5D64265BAAFF1103F54DBF5
2016-05-17 22:37:38,553 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:38,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A9A8D577C5D64265BAAFF1103F54DBF5
2016-05-17 22:37:38,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A9A8D577C5D64265BAAFF1103F54DBF5 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:38,555 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:38,555 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:38,555 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:38,556 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524658255_0001_1_00_000000_0
2016-05-17 22:37:38,560 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:38,562 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: A9A8D577C5D64265BAAFF1103F54DBF5
2016-05-17 22:37:38,562 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1"], id: A93506A481F7459883BAC162B887411C
2016-05-17 22:37:38,564 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1"]
2016-05-17 22:37:38,565 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:38,565 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:38,565 INFO input.MRInput (MRInput.java:checkAndAwaitRecordReaderInitialization(597)) - Awaiting RecordReader initialization
2016-05-17 22:37:38,566 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:38,567 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@676e42ea
2016-05-17 22:37:38,572 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@676e42ea
2016-05-17 22:37:38,572 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:38,572 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:38,573 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C89F90374A32436DBFA862B447C09DF1, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:38,576 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:38,577 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:38,577 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524658255_0001_1_00_000000_0
2016-05-17 22:37:38,577 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524658255_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:38,578 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524658255_0001_1_00_000000_0
2016-05-17 22:37:38,579 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C89F90374A32436DBFA862B447C09DF1, mem on close (mb), free: 320, total: 383, max: 672
2016-05-17 22:37:38 Completed running task attempt: attempt_1463524658255_0001_1_00_000000_0
2016-05-17 22:37:38,580 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524658255_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:38,583 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:38,583 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524658255_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C89F90374A32436DBFA862B447C09DF1, taskAttemptId=attempt_1463524658255_0001_1_00_000000_0, startTime=1463524658507, finishTime=1463524658583, timeTaken=76, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524658573, Process_Duration=5, Process_End_Time=1463524658578, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:38,584 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524658255_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:38,584 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:38,584 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524658255_0001_1_00_000000_0
2016-05-17 22:37:38,585 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:38,585 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524658255_0001_1][Event:TASK_FINISHED]: vertexName=C89F90374A32436DBFA862B447C09DF1, taskId=task_1463524658255_0001_1_00_000000, startTime=1463524658507, finishTime=1463524658584, timeTaken=77, status=SUCCEEDED, successfulAttemptID=attempt_1463524658255_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524658573, Process_Duration=5, Process_End_Time=1463524658578, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:38,585 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524658255_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:38,586 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1] : 1
2016-05-17 22:37:38,586 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1]
2016-05-17 22:37:38,588 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:38,589 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524658255_0001_1][Event:VERTEX_FINISHED]: vertexName=C89F90374A32436DBFA862B447C09DF1, vertexId=vertex_1463524658255_0001_1_00, initRequestedTime=1463524658458, initedTime=1463524658482, startRequestedTime=1463524658483, startedTime=1463524658483, finishTime=1463524658586, timeTaken=103, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524658573, Process_Duration=5, Process_End_Time=1463524658578, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524658507, firstTasksToStart=[ task_1463524658255_0001_1_00_000000 ], lastTaskFinishTime=1463524658583, lastTasksToFinish=[ task_1463524658255_0001_1_00_000000 ], minTaskDuration=76, maxTaskDuration=76, avgTaskDuration=76.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524658255_0001_1_00_000000 ], longestDurationTasks=[ task_1463524658255_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:38,589 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:38,590 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524658573, Process_Duration=5, Process_End_Time=1463524658578, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:38,590 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524658255_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:38,591 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:38,592 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:38,591 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524658255_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524658255_0001_00_000001, stoppedTime=1463524658591, exitStatus=0
2016-05-17 22:37:38,592 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:38,592 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524658255_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:38,593 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:38,593 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524658255_0001_1
2016-05-17 22:37:38,594 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524658255_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524658255_0001_1
2016-05-17 22:37:38,594 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A93506A481F7459883BAC162B887411C for vertex: vertex_1463524658255_0001_1_00 [C89F90374A32436DBFA862B447C09DF1]
2016-05-17 22:37:38,597 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524658255_0001_00_000001
2016-05-17 22:37:38,598 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524658255_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:38,598 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524658255_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:38,598 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524658255_0001_00_000001, Exiting
2016-05-17 22:37:38,598 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524658255_0001_00_000001
2016-05-17 22:37:38,598 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524658255_0001_00_000001
2016-05-17 22:37:38,604 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524658255_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:38,605 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524658255_0001_00_000001 completed successfully
2016-05-17 22:37:38,614 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524658255_0001_1, queueSize=0
2016-05-17 22:37:38,614 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:38,614 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524658255_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524658255_0001_1, startTime=1463524658456, finishTime=1463524658612, timeTaken=156, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524658573, Process_Duration=5, Process_End_Time=1463524658578, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:38,614 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524658255_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:38,615 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524658255_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:38,615 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524658255_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:38 Completed Dag: dag_1463524658255_0001_1
2016-05-17 22:37:38,615 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524658255_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:38,615 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524658255_0001_1_post
2016-05-17 22:37:38,616 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:38,616 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:38,616 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524658255_0001_00_000001
2016-05-17 22:37:38,624 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.-1_UNKNOWN_-1, applicationId=application_1463524658255_0001
2016-05-17 22:37:38,630 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:38,630 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:38,630 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:38,630 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:38,630 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:38,631 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:38,635 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:38,635 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:38,636 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:38,637 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:38,637 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:38,638 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:38,640 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:38,643 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ADDAD650261D476C8ACF2B1208BFFC1B/D258FD57E81C4CD8B523F3B05845A76F/.tez/application_1463524658255_0001
2016-05-17 22:37:38,644 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_-1/_temporary
2016-05-17 22:37:38,645 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.441
2016-05-17 22:37:38,753 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_-1
2016-05-17 22:37:38,788 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:38,789 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:38,839 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.009
2016-05-17 22:37:38,843 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.014
2016-05-17 22:37:38,843 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:38,843 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:38,844 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_'field2'
2016-05-17 22:37:38,879 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:38,880 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:38,937 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 22:37:38,951 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.032
2016-05-17 22:37:38,952 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:38,952 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:38,953 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:38,977 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:38,978 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'
2016-05-17 22:37:39,000 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:39,001 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:39,003 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'"]
2016-05-17 22:37:39,003 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:39,004 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:39,004 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:39,005 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-17 22:37:39,005 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:39,026 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:39,032 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:39,032 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:39,032 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:39,046 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3323428E834E4C14BD5614178D8FFF39/D4A523D81F0D4652B900D450D588A9FB/.tez/application_1463524659032_0001 doesn't exist and is created
2016-05-17 22:37:39,076 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:39,077 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:39,078 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3323428E834E4C14BD5614178D8FFF39/D4A523D81F0D4652B900D450D588A9FB/.tez/application_1463524659032_0001
2016-05-17 22:37:39,084 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524659032_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:39,106 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:39,108 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:39,111 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:39,111 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:39,112 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3323428E834E4C14BD5614178D8FFF39/D4A523D81F0D4652B900D450D588A9FB/.tez/application_1463524659032_0001/localmode-log-dir
2016-05-17 22:37:39,113 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3323428E834E4C14BD5614178D8FFF39/D4A523D81F0D4652B900D450D588A9FB/.tez/application_1463524659032_0001/localmode-log-dir/history.txt.appattempt_1463524659032_0001_000000, maxErrors=10
2016-05-17 22:37:39,113 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:39,114 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524659032_0001_000000, appSubmitTime=1463524659083, launchTime=1463524659084
2016-05-17 22:37:39,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:39,121 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:39,122 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:39,122 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:39,132 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:44037
2016-05-17 22:37:39,133 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:39,138 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524659032_0001_000000, startTime=1463524659138
2016-05-17 22:37:39,138 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:39,138 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:39,177 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:39,177 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:39,177 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_'field', applicationId=application_1463524659032_0001, dagName=[3323428E834E4C14BD5614178D8FFF39/D4A523D81F0D4652B900D450D588A9FB] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-17 22:37:39,202 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3323428E834E4C14BD5614178D8FFF39/D4A523D81F0D4652B900D450D588A9FB] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-17 22:37:39,204 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524659032_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3323428E834E4C14BD5614178D8FFF39/D4A523D81F0D4652B900D450D588A9FB/.tez/application_1463524659032_0001/localmode-log-dir/dag_1463524659032_0001_1.dot
2016-05-17 22:37:39,206 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3323428E834E4C14BD5614178D8FFF39/D4A523D81F0D4652B900D450D588A9FB/.tez/application_1463524659032_0001/localmode-log-dir/dag_1463524659032_0001_1-tez-dag.pb.txt
2016-05-17 22:37:39,218 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524659032_0001_1
2016-05-17 22:37:39,218 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:39,218 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3323428E834E4C14BD5614178D8FFF39/D4A523D81F0D4652B900D450D588A9FB] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-17 22:37:39 Running Dag: dag_1463524659032_0001_1
2016-05-17 22:37:39,233 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:39,234 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659032_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524659032_0001_1, submitTime=1463524659202
2016-05-17 22:37:39,234 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F5B9F5F9591B4D36B82F0235D7A1353F
2016-05-17 22:37:39,235 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:39,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:39,236 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659032_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524659032_0001_1, initTime=1463524659234
2016-05-17 22:37:39,236 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524659032_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:39,237 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659032_0001_1][Event:DAG_STARTED]: dagID=dag_1463524659032_0001_1, startTime=1463524659237
2016-05-17 22:37:39,237 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:39,237 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:39,237 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524659032_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:39,238 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F5B9F5F9591B4D36B82F0235D7A1353F : {1C9C963C02ED422F9E6D9AC968B7F5E9={InputName=1C9C963C02ED422F9E6D9AC968B7F5E9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:39,238 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 1C9C963C02ED422F9E6D9AC968B7F5E9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:39,238 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F]
2016-05-17 22:37:39,238 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F]
2016-05-17 22:37:39,238 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F]
2016-05-17 22:37:39,239 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F]. Starting root input initializers: 1
2016-05-17 22:37:39,239 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:39,239 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_'field', applicationId=application_1463524659032_0001, dagName=[3323428E834E4C14BD5614178D8FFF39/D4A523D81F0D4652B900D450D588A9FB] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field'
2016-05-17 22:37:39,240 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 1C9C963C02ED422F9E6D9AC968B7F5E9 on vertex vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F]
2016-05-17 22:37:39,241 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:39,241 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 1C9C963C02ED422F9E6D9AC968B7F5E9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:39,242 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:39,242 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:39,244 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:39,245 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 1C9C963C02ED422F9E6D9AC968B7F5E9 on vertex vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F]
2016-05-17 22:37:39,245 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:39,245 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {1C9C963C02ED422F9E6D9AC968B7F5E9=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:39,245 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F] parallelism set to 1
2016-05-17 22:37:39,245 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:39,246 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659032_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524659032_0001_1_00, eventCount=1
2016-05-17 22:37:39,246 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F]
2016-05-17 22:37:39,246 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2B30742957C0495F9ADDB684A486DFD7, vertexId=vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:39,247 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2B30742957C0495F9ADDB684A486DFD7, vertexId=vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F]
2016-05-17 22:37:39,249 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2B30742957C0495F9ADDB684A486DFD7, vertexName=F5B9F5F9591B4D36B82F0235D7A1353F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:39,249 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2B30742957C0495F9ADDB684A486DFD7, vertexId=vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F]
2016-05-17 22:37:39,250 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:39,250 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659032_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F5B9F5F9591B4D36B82F0235D7A1353F, vertexId=vertex_1463524659032_0001_1_00, initRequestedTime=1463524659238, initedTime=1463524659250, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:39,251 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F] with distanceFromRoot: 0
2016-05-17 22:37:39,251 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:39,251 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F5B9F5F9591B4D36B82F0235D7A1353F
2016-05-17 22:37:39,251 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F]
2016-05-17 22:37:39,251 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:39,251 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659032_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524659032_0001_1_00, startRequestedTime=1463524659251, startedTime=1463524659251
2016-05-17 22:37:39,252 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:39,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:39,252 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659032_0001_1][Event:TASK_STARTED]: vertexName=F5B9F5F9591B4D36B82F0235D7A1353F, taskId=task_1463524659032_0001_1_00_000000, scheduledTime=1463524659252, launchTime=1463524659252
2016-05-17 22:37:39,252 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524659032_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:39,253 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F]
2016-05-17 22:37:39,253 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3323428E834E4C14BD5614178D8FFF39/D4A523D81F0D4652B900D450D588A9FB] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field', VertexName: F5B9F5F9591B4D36B82F0235D7A1353F, VertexParallelism: 1, TaskAttemptID:attempt_1463524659032_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1C9C963C02ED422F9E6D9AC968B7F5E9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=2B30742957C0495F9ADDB684A486DFD7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:39,253 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524659032_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:39,253 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:39,254 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:39,255 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:39,255 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:39,255 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524659032_0001_00_000001
2016-05-17 22:37:39,256 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524659032_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:39,257 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524659032_0001_00_000001
2016-05-17 22:37:39,257 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:39,258 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524659032_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:39,258 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:39,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659032_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524659032_0001_00_000001, launchTime=1463524659258
2016-05-17 22:37:39,260 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524659032_0001_00_000001
2016-05-17 22:37:39,261 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524659032_0001_1_00_000000_0] to container: [container_1463524659032_0001_00_000001]
2016-05-17 22:37:39,261 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524659032_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:39,261 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524659032_0001_00_000001 given task: attempt_1463524659032_0001_1_00_000000_0
2016-05-17 22:37:39,261 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524659032_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524659032_0001_1_00_000000_0
2016-05-17 22:37:39,261 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524659032_0001_1_00_000000_0] started. Is using containerId: [container_1463524659032_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:39,262 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659032_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F5B9F5F9591B4D36B82F0235D7A1353F, taskAttemptId=attempt_1463524659032_0001_1_00_000000_0, startTime=1463524659261, containerId=container_1463524659032_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524659032_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:39,262 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524659032_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:39,263 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:39,263 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524659032_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:39 Starting to run new task attempt: attempt_1463524659032_0001_1_00_000000_0
2016-05-17 22:37:39,263 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524659032_0001_1_00_000000_0
2016-05-17 22:37:39,263 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:39,263 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:39,264 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:39,264 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:39,264 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:39,264 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:39,264 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3323428E834E4C14BD5614178D8FFF39/D4A523D81F0D4652B900D450D588A9FB] every.every/(1/1) ...ESULTS.-1_UNKNOWN_'field', VertexName: F5B9F5F9591B4D36B82F0235D7A1353F, VertexParallelism: 1, TaskAttemptID:attempt_1463524659032_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1C9C963C02ED422F9E6D9AC968B7F5E9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=2B30742957C0495F9ADDB684A486DFD7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:39,264 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:39,265 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524659032_0001_1_00_000000_0
2016-05-17 22:37:39,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:39,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:39,266 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524659032_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:39,267 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=1C9C963C02ED422F9E6D9AC968B7F5E9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:39,267 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2B30742957C0495F9ADDB684A486DFD7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:39,268 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:39,268 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 1C9C963C02ED422F9E6D9AC968B7F5E9
2016-05-17 22:37:39,268 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 1C9C963C02ED422F9E6D9AC968B7F5E9
2016-05-17 22:37:39,267 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:39,267 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524659032_0001, with dag id: dag_1463524659032_0001_1
2016-05-17 22:37:39,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:39,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2B30742957C0495F9ADDB684A486DFD7
2016-05-17 22:37:39,273 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:39,273 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2B30742957C0495F9ADDB684A486DFD7
2016-05-17 22:37:39,275 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:39,280 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:39,280 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 1C9C963C02ED422F9E6D9AC968B7F5E9
2016-05-17 22:37:39,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 1C9C963C02ED422F9E6D9AC968B7F5E9
2016-05-17 22:37:39,286 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:39,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2B30742957C0495F9ADDB684A486DFD7
2016-05-17 22:37:39,290 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F5B9F5F9591B4D36B82F0235D7A1353F, ordinal: 0
2016-05-17 22:37:39,290 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F5B9F5F9591B4D36B82F0235D7A1353F, mem on start (mb), free: 229, total: 383, max: 672
2016-05-17 22:37:39,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:39,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:39,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:39,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:39,291 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:39,291 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:39,291 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:39,291 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:39,291 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:39,291 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:39,292 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:39,292 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 1C9C963C02ED422F9E6D9AC968B7F5E9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:39,292 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2B30742957C0495F9ADDB684A486DFD7, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:39,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:39,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 1C9C963C02ED422F9E6D9AC968B7F5E9
2016-05-17 22:37:39,292 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:39,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 1C9C963C02ED422F9E6D9AC968B7F5E9
2016-05-17 22:37:39,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 1C9C963C02ED422F9E6D9AC968B7F5E9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:39,293 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:39,293 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:39,293 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:39,293 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524659032_0001_1_00_000000_0
2016-05-17 22:37:39,296 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:39,296 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 1C9C963C02ED422F9E6D9AC968B7F5E9
2016-05-17 22:37:39,297 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'"], id: 2B30742957C0495F9ADDB684A486DFD7
2016-05-17 22:37:39,297 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'"]
2016-05-17 22:37:39,297 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:39,297 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:39,297 INFO input.MRInput (MRInput.java:checkAndAwaitRecordReaderInitialization(597)) - Awaiting RecordReader initialization
2016-05-17 22:37:39,298 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:39,298 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@13b2bf3f
2016-05-17 22:37:39,299 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@13b2bf3f
2016-05-17 22:37:39,299 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:39,299 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:39,300 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F5B9F5F9591B4D36B82F0235D7A1353F, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:39,302 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:39,303 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:39,303 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524659032_0001_1_00_000000_0
2016-05-17 22:37:39,303 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524659032_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:39,304 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524659032_0001_1_00_000000_0
2016-05-17 22:37:39,304 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F5B9F5F9591B4D36B82F0235D7A1353F, mem on close (mb), free: 227, total: 383, max: 672
2016-05-17 22:37:39 Completed running task attempt: attempt_1463524659032_0001_1_00_000000_0
2016-05-17 22:37:39,305 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524659032_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:39,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524659300, Process_Duration=4, Process_End_Time=1463524659304, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:39,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659032_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F5B9F5F9591B4D36B82F0235D7A1353F, taskAttemptId=attempt_1463524659032_0001_1_00_000000_0, startTime=1463524659261, finishTime=1463524659306, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524659300, Process_Duration=4, Process_End_Time=1463524659304, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:39,307 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:39,307 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524659032_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:39,308 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:39,308 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524659032_0001_1_00_000000_0
2016-05-17 22:37:39,309 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:39,310 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659032_0001_1][Event:TASK_FINISHED]: vertexName=F5B9F5F9591B4D36B82F0235D7A1353F, taskId=task_1463524659032_0001_1_00_000000, startTime=1463524659261, finishTime=1463524659309, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1463524659032_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524659300, Process_Duration=4, Process_End_Time=1463524659304, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:39,310 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524659032_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:39,308 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:39,310 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:39,310 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F] : 1
2016-05-17 22:37:39,310 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524659032_0001_00_000001
2016-05-17 22:37:39,310 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F]
2016-05-17 22:37:39,312 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:39,312 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659032_0001_1][Event:VERTEX_FINISHED]: vertexName=F5B9F5F9591B4D36B82F0235D7A1353F, vertexId=vertex_1463524659032_0001_1_00, initRequestedTime=1463524659238, initedTime=1463524659250, startRequestedTime=1463524659251, startedTime=1463524659251, finishTime=1463524659311, timeTaken=60, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524659300, Process_Duration=4, Process_End_Time=1463524659304, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524659261, firstTasksToStart=[ task_1463524659032_0001_1_00_000000 ], lastTaskFinishTime=1463524659306, lastTasksToFinish=[ task_1463524659032_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524659032_0001_1_00_000000 ], longestDurationTasks=[ task_1463524659032_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:39,312 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:39,313 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524659032_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:39,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659032_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524659032_0001_00_000001, stoppedTime=1463524659313, exitStatus=0
2016-05-17 22:37:39,313 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:39,313 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524659032_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:39,313 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:39,313 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524659032_0001_1
2016-05-17 22:37:39,314 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659032_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524659032_0001_1
2016-05-17 22:37:39,314 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2B30742957C0495F9ADDB684A486DFD7 for vertex: vertex_1463524659032_0001_1_00 [F5B9F5F9591B4D36B82F0235D7A1353F]
2016-05-17 22:37:39,321 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524659032_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:39,321 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524659032_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:39,321 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524659032_0001_00_000001, Exiting
2016-05-17 22:37:39,321 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524659032_0001_00_000001
2016-05-17 22:37:39,321 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524659032_0001_00_000001
2016-05-17 22:37:39,326 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524659032_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:39,326 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524659032_0001_00_000001 completed successfully
2016-05-17 22:37:39,328 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524659032_0001_1, queueSize=0
2016-05-17 22:37:39,328 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:39,328 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659032_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524659032_0001_1, startTime=1463524659237, finishTime=1463524659321, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524659300, Process_Duration=4, Process_End_Time=1463524659304, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:39,328 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524659032_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:39,329 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524659032_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:39,330 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524659032_0001_00_000001
2016-05-17 22:37:39,331 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524659032_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:39 Completed Dag: dag_1463524659032_0001_1
2016-05-17 22:37:39,332 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524659032_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:39,333 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524659032_0001_1_post
2016-05-17 22:37:39,333 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:39,333 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:39,338 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_'field', applicationId=application_1463524659032_0001
2016-05-17 22:37:39,341 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:39,341 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:39,341 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:39,341 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:39,341 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:39,342 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:39,342 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:39,343 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:39,343 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:39,343 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:39,344 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:39,344 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:39,345 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:39,348 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3323428E834E4C14BD5614178D8FFF39/D4A523D81F0D4652B900D450D588A9FB/.tez/application_1463524659032_0001
2016-05-17 22:37:39,349 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_'field'/_temporary
2016-05-17 22:37:39,349 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.348
2016-05-17 22:37:39,436 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_'field'
2016-05-17 22:37:39,471 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:39,472 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:39,527 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 22:37:39,536 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-17 22:37:39,537 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:39,537 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:39,537 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:39,553 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:39,554 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS
2016-05-17 22:37:39,567 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:39,567 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:39,568 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS"]
2016-05-17 22:37:39,568 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:39,568 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:39,569 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:39,570 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-17 22:37:39,570 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:39,583 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:39,588 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:39,588 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:39,589 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:39,598 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1DCC1A9D41C84BD5B325077BFE945DD4/A6AD3CEA948C40198034A7830D96103B/.tez/application_1463524659588_0001 doesn't exist and is created
2016-05-17 22:37:39,617 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:39,617 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:39,618 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1DCC1A9D41C84BD5B325077BFE945DD4/A6AD3CEA948C40198034A7830D96103B/.tez/application_1463524659588_0001
2016-05-17 22:37:39,622 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524659588_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:39,633 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:39,634 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:39,635 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:39,636 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:39,636 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1DCC1A9D41C84BD5B325077BFE945DD4/A6AD3CEA948C40198034A7830D96103B/.tez/application_1463524659588_0001/localmode-log-dir
2016-05-17 22:37:39,636 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1DCC1A9D41C84BD5B325077BFE945DD4/A6AD3CEA948C40198034A7830D96103B/.tez/application_1463524659588_0001/localmode-log-dir/history.txt.appattempt_1463524659588_0001_000000, maxErrors=10
2016-05-17 22:37:39,636 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:39,637 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524659588_0001_000000, appSubmitTime=1463524659621, launchTime=1463524659622
2016-05-17 22:37:39,640 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:39,641 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:39,642 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:39,642 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:39,644 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:51620
2016-05-17 22:37:39,647 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:39,651 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524659588_0001_000000, startTime=1463524659651
2016-05-17 22:37:39,651 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:39,652 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:39,718 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:39,718 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:39,718 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, applicationId=application_1463524659588_0001, dagName=[1DCC1A9D41C84BD5B325077BFE945DD4/A6AD3CEA948C40198034A7830D96103B] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-17 22:37:39,735 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1DCC1A9D41C84BD5B325077BFE945DD4/A6AD3CEA948C40198034A7830D96103B] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-17 22:37:39,736 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524659588_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1DCC1A9D41C84BD5B325077BFE945DD4/A6AD3CEA948C40198034A7830D96103B/.tez/application_1463524659588_0001/localmode-log-dir/dag_1463524659588_0001_1.dot
2016-05-17 22:37:39,737 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1DCC1A9D41C84BD5B325077BFE945DD4/A6AD3CEA948C40198034A7830D96103B/.tez/application_1463524659588_0001/localmode-log-dir/dag_1463524659588_0001_1-tez-dag.pb.txt
2016-05-17 22:37:39,750 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524659588_0001_1
2016-05-17 22:37:39,750 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:39,750 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1DCC1A9D41C84BD5B325077BFE945DD4/A6AD3CEA948C40198034A7830D96103B] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-17 22:37:39 Running Dag: dag_1463524659588_0001_1
2016-05-17 22:37:39,764 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659588_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524659588_0001_1, submitTime=1463524659735
2016-05-17 22:37:39,764 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:39,765 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A96C1AA641C84A37BEB3F337AB5E0B96
2016-05-17 22:37:39,766 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:39,767 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:39,767 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659588_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524659588_0001_1, initTime=1463524659765
2016-05-17 22:37:39,767 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524659588_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:39,768 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659588_0001_1][Event:DAG_STARTED]: dagID=dag_1463524659588_0001_1, startTime=1463524659768
2016-05-17 22:37:39,768 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:39,768 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:39,769 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524659588_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:39,769 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A96C1AA641C84A37BEB3F337AB5E0B96 : {025FE7B13A614F6799672786DE379B8A={InputName=025FE7B13A614F6799672786DE379B8A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:39,769 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 025FE7B13A614F6799672786DE379B8A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:39,769 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96]
2016-05-17 22:37:39,769 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96]
2016-05-17 22:37:39,769 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96]
2016-05-17 22:37:39,770 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96]. Starting root input initializers: 1
2016-05-17 22:37:39,770 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, applicationId=application_1463524659588_0001, dagName=[1DCC1A9D41C84BD5B325077BFE945DD4/A6AD3CEA948C40198034A7830D96103B] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS
2016-05-17 22:37:39,772 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:39,773 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 025FE7B13A614F6799672786DE379B8A on vertex vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96]
2016-05-17 22:37:39,775 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:39,775 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 025FE7B13A614F6799672786DE379B8A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:39,776 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:39,777 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:39,780 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:39,780 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 025FE7B13A614F6799672786DE379B8A on vertex vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96]
2016-05-17 22:37:39,781 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:39,781 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {025FE7B13A614F6799672786DE379B8A=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:39,781 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96] parallelism set to 1
2016-05-17 22:37:39,781 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:39,781 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659588_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524659588_0001_1_00, eventCount=1
2016-05-17 22:37:39,781 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96]
2016-05-17 22:37:39,782 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D7C0D7338EAE4A93AD49F7A3C62C06E4, vertexId=vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:39,782 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D7C0D7338EAE4A93AD49F7A3C62C06E4, vertexId=vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96]
2016-05-17 22:37:39,784 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D7C0D7338EAE4A93AD49F7A3C62C06E4, vertexName=A96C1AA641C84A37BEB3F337AB5E0B96, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:39,784 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D7C0D7338EAE4A93AD49F7A3C62C06E4, vertexId=vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96]
2016-05-17 22:37:39,786 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524659588_0001, with dag id: dag_1463524659588_0001_1
2016-05-17 22:37:39,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:39,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659588_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A96C1AA641C84A37BEB3F337AB5E0B96, vertexId=vertex_1463524659588_0001_1_00, initRequestedTime=1463524659769, initedTime=1463524659786, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:39,787 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96] with distanceFromRoot: 0
2016-05-17 22:37:39,788 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:39,788 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A96C1AA641C84A37BEB3F337AB5E0B96
2016-05-17 22:37:39,788 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96]
2016-05-17 22:37:39,789 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:39,790 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659588_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524659588_0001_1_00, startRequestedTime=1463524659788, startedTime=1463524659788
2016-05-17 22:37:39,790 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:39,791 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:39,792 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659588_0001_1][Event:TASK_STARTED]: vertexName=A96C1AA641C84A37BEB3F337AB5E0B96, taskId=task_1463524659588_0001_1_00_000000, scheduledTime=1463524659791, launchTime=1463524659791
2016-05-17 22:37:39,792 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524659588_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:39,792 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96]
2016-05-17 22:37:39,793 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1DCC1A9D41C84BD5B325077BFE945DD4/A6AD3CEA948C40198034A7830D96103B] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, VertexName: A96C1AA641C84A37BEB3F337AB5E0B96, VertexParallelism: 1, TaskAttemptID:attempt_1463524659588_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=025FE7B13A614F6799672786DE379B8A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D7C0D7338EAE4A93AD49F7A3C62C06E4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:39,793 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524659588_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:39,793 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:39,795 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:39,797 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:39,797 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:39,798 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524659588_0001_00_000001
2016-05-17 22:37:39,798 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524659588_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:39,800 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524659588_0001_00_000001
2016-05-17 22:37:39,800 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:39,800 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524659588_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:39,801 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:39,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659588_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524659588_0001_00_000001, launchTime=1463524659801
2016-05-17 22:37:39,803 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524659588_0001_00_000001
2016-05-17 22:37:39,803 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524659588_0001_1_00_000000_0] to container: [container_1463524659588_0001_00_000001]
2016-05-17 22:37:39,803 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524659588_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:39,804 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524659588_0001_00_000001 given task: attempt_1463524659588_0001_1_00_000000_0
2016-05-17 22:37:39,804 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524659588_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524659588_0001_1_00_000000_0
2016-05-17 22:37:39,804 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524659588_0001_1_00_000000_0] started. Is using containerId: [container_1463524659588_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:39,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659588_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A96C1AA641C84A37BEB3F337AB5E0B96, taskAttemptId=attempt_1463524659588_0001_1_00_000000_0, startTime=1463524659804, containerId=container_1463524659588_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524659588_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:39,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:39,804 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524659588_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:39 Starting to run new task attempt: attempt_1463524659588_0001_1_00_000000_0
2016-05-17 22:37:39,805 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524659588_0001_1_00_000000_0
2016-05-17 22:37:39,805 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:39,805 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:39,805 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:39,805 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:39,806 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:39,806 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:39,806 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1DCC1A9D41C84BD5B325077BFE945DD4/A6AD3CEA948C40198034A7830D96103B] every.every/(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, VertexName: A96C1AA641C84A37BEB3F337AB5E0B96, VertexParallelism: 1, TaskAttemptID:attempt_1463524659588_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=025FE7B13A614F6799672786DE379B8A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D7C0D7338EAE4A93AD49F7A3C62C06E4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:39,806 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:39,807 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524659588_0001_1_00_000000_0
2016-05-17 22:37:39,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:39,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:39,805 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524659588_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:39,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=025FE7B13A614F6799672786DE379B8A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:39,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:39,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 025FE7B13A614F6799672786DE379B8A
2016-05-17 22:37:39,810 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 025FE7B13A614F6799672786DE379B8A
2016-05-17 22:37:39,809 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524659588_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:39,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D7C0D7338EAE4A93AD49F7A3C62C06E4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:39,810 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:39,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:39,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D7C0D7338EAE4A93AD49F7A3C62C06E4
2016-05-17 22:37:39,812 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:39,812 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D7C0D7338EAE4A93AD49F7A3C62C06E4
2016-05-17 22:37:39,816 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:39,816 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 025FE7B13A614F6799672786DE379B8A
2016-05-17 22:37:39,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 025FE7B13A614F6799672786DE379B8A
2016-05-17 22:37:39,818 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:39,826 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:39,826 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A96C1AA641C84A37BEB3F337AB5E0B96, ordinal: 0
2016-05-17 22:37:39,826 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D7C0D7338EAE4A93AD49F7A3C62C06E4
2016-05-17 22:37:39,826 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A96C1AA641C84A37BEB3F337AB5E0B96, mem on start (mb), free: 150, total: 383, max: 672
2016-05-17 22:37:39,826 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:39,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:39,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:39,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:39,827 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:39,827 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:39,827 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:39,828 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:39,828 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:39,828 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:39,829 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:39,829 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 025FE7B13A614F6799672786DE379B8A, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:39,829 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D7C0D7338EAE4A93AD49F7A3C62C06E4, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:39,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:39,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 025FE7B13A614F6799672786DE379B8A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:39,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 025FE7B13A614F6799672786DE379B8A
2016-05-17 22:37:39,830 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:39,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 025FE7B13A614F6799672786DE379B8A
2016-05-17 22:37:39,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:39,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:39,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:39,831 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524659588_0001_1_00_000000_0
2016-05-17 22:37:39,835 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:39,835 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:39,836 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 025FE7B13A614F6799672786DE379B8A
2016-05-17 22:37:39,836 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS"], id: D7C0D7338EAE4A93AD49F7A3C62C06E4
2016-05-17 22:37:39,836 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS"]
2016-05-17 22:37:39,836 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@72688124
2016-05-17 22:37:39,837 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:39,837 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:39,838 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@72688124
2016-05-17 22:37:39,838 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:39,838 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:39,839 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A96C1AA641C84A37BEB3F337AB5E0B96, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:39,841 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:39,841 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:39,842 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524659588_0001_1_00_000000_0
2016-05-17 22:37:39,842 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524659588_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:39,843 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524659588_0001_1_00_000000_0
2016-05-17 22:37:39,843 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A96C1AA641C84A37BEB3F337AB5E0B96, mem on close (mb), free: 148, total: 383, max: 672
2016-05-17 22:37:39 Completed running task attempt: attempt_1463524659588_0001_1_00_000000_0
2016-05-17 22:37:39,844 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524659588_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:39,845 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:39,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659588_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A96C1AA641C84A37BEB3F337AB5E0B96, taskAttemptId=attempt_1463524659588_0001_1_00_000000_0, startTime=1463524659804, finishTime=1463524659845, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524659839, Process_Duration=4, Process_End_Time=1463524659843, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:39,846 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524659588_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:39,846 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524659588_0001_1_00_000000_0
2016-05-17 22:37:39,846 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:39,847 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:39,847 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659588_0001_1][Event:TASK_FINISHED]: vertexName=A96C1AA641C84A37BEB3F337AB5E0B96, taskId=task_1463524659588_0001_1_00_000000, startTime=1463524659804, finishTime=1463524659846, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463524659588_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524659839, Process_Duration=4, Process_End_Time=1463524659843, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:39,847 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524659588_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:39,848 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96] : 1
2016-05-17 22:37:39,848 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96]
2016-05-17 22:37:39,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:39,850 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659588_0001_1][Event:VERTEX_FINISHED]: vertexName=A96C1AA641C84A37BEB3F337AB5E0B96, vertexId=vertex_1463524659588_0001_1_00, initRequestedTime=1463524659769, initedTime=1463524659786, startRequestedTime=1463524659788, startedTime=1463524659788, finishTime=1463524659848, timeTaken=60, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524659839, Process_Duration=4, Process_End_Time=1463524659843, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524659804, firstTasksToStart=[ task_1463524659588_0001_1_00_000000 ], lastTaskFinishTime=1463524659845, lastTasksToFinish=[ task_1463524659588_0001_1_00_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524659588_0001_1_00_000000 ], longestDurationTasks=[ task_1463524659588_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:39,850 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:39,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524659839, Process_Duration=4, Process_End_Time=1463524659843, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:39,851 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524659588_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:39,851 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:39,853 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:39,853 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524659588_0001_00_000001
2016-05-17 22:37:39,853 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659588_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524659588_0001_00_000001, stoppedTime=1463524659853, exitStatus=0
2016-05-17 22:37:39,853 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524659588_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:39,853 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524659588_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:39,853 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524659588_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:39,853 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:39,853 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524659588_0001_00_000001, Exiting
2016-05-17 22:37:39,853 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524659588_0001_00_000001
2016-05-17 22:37:39,854 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524659588_0001_00_000001
2016-05-17 22:37:39,853 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:39,855 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524659588_0001_1
2016-05-17 22:37:39,856 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659588_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524659588_0001_1
2016-05-17 22:37:39,856 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D7C0D7338EAE4A93AD49F7A3C62C06E4 for vertex: vertex_1463524659588_0001_1_00 [A96C1AA641C84A37BEB3F337AB5E0B96]
2016-05-17 22:37:39,857 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524659588_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:39,857 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524659588_0001_00_000001 completed successfully
2016-05-17 22:37:39,865 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524659588_0001_1, queueSize=0
2016-05-17 22:37:39,866 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:39,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524659588_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524659588_0001_1, startTime=1463524659768, finishTime=1463524659864, timeTaken=96, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=401604608, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524659839, Process_Duration=4, Process_End_Time=1463524659843, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:39,867 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524659588_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:39,868 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524659588_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:39,868 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524659588_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:39,868 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524659588_0001_00_000001
2016-05-17 22:37:39 Completed Dag: dag_1463524659588_0001_1
2016-05-17 22:37:39,870 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524659588_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:39,871 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524659588_0001_1_post
2016-05-17 22:37:39,871 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:39,872 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:39,875 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ESULTS.-1_UNKNOWN_RESULTS, applicationId=application_1463524659588_0001
2016-05-17 22:37:39,877 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:39,877 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:39,877 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:39,877 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:39,878 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:39,878 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:39,879 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:39,879 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:39,879 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:39,879 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:39,879 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:39,879 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:39,879 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:39,883 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1DCC1A9D41C84BD5B325077BFE945DD4/A6AD3CEA948C40198034A7830D96103B/.tez/application_1463524659588_0001
2016-05-17 22:37:39,884 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_RESULTS/_temporary
2016-05-17 22:37:39,885 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.317
2016-05-17 22:37:39,981 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_RESULTS
2016-05-17 22:37:40,006 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:40,006 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:40,051 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 22:37:40,064 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.031
2016-05-17 22:37:40,065 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:40,065 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:40,065 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:40,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:40,080 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL
2016-05-17 22:37:40,091 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:40,092 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:40,092 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL"]
2016-05-17 22:37:40,092 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:40,092 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:40,092 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:40,093 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-17 22:37:40,093 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:40,106 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:40,111 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:40,111 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:40,112 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:40,121 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/38601E3A8065470E8EB8A9805E879697/ABD7B3287E2543E081B3E632C7B34314/.tez/application_1463524660111_0001 doesn't exist and is created
2016-05-17 22:37:40,139 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:40,139 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:40,139 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/38601E3A8065470E8EB8A9805E879697/ABD7B3287E2543E081B3E632C7B34314/.tez/application_1463524660111_0001
2016-05-17 22:37:40,144 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524660111_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:40,155 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:40,156 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:40,157 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:40,158 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:40,158 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/38601E3A8065470E8EB8A9805E879697/ABD7B3287E2543E081B3E632C7B34314/.tez/application_1463524660111_0001/localmode-log-dir
2016-05-17 22:37:40,158 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/38601E3A8065470E8EB8A9805E879697/ABD7B3287E2543E081B3E632C7B34314/.tez/application_1463524660111_0001/localmode-log-dir/history.txt.appattempt_1463524660111_0001_000000, maxErrors=10
2016-05-17 22:37:40,158 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:40,159 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524660111_0001_000000, appSubmitTime=1463524660142, launchTime=1463524660143
2016-05-17 22:37:40,168 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:40,168 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:40,170 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:40,169 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53666
2016-05-17 22:37:40,171 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:40,174 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:40,178 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524660111_0001_000000, startTime=1463524660178
2016-05-17 22:37:40,178 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:40,178 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:40,239 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:40,239 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:40,239 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, applicationId=application_1463524660111_0001, dagName=[38601E3A8065470E8EB8A9805E879697/ABD7B3287E2543E081B3E632C7B34314] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-17 22:37:40,255 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [38601E3A8065470E8EB8A9805E879697/ABD7B3287E2543E081B3E632C7B34314] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-17 22:37:40,257 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524660111_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/38601E3A8065470E8EB8A9805E879697/ABD7B3287E2543E081B3E632C7B34314/.tez/application_1463524660111_0001/localmode-log-dir/dag_1463524660111_0001_1.dot
2016-05-17 22:37:40,258 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/38601E3A8065470E8EB8A9805E879697/ABD7B3287E2543E081B3E632C7B34314/.tez/application_1463524660111_0001/localmode-log-dir/dag_1463524660111_0001_1-tez-dag.pb.txt
2016-05-17 22:37:40,264 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524660111_0001_1
2016-05-17 22:37:40,264 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:40,265 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [38601E3A8065470E8EB8A9805E879697/ABD7B3287E2543E081B3E632C7B34314] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-17 22:37:40 Running Dag: dag_1463524660111_0001_1
2016-05-17 22:37:40,277 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:40,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660111_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524660111_0001_1, submitTime=1463524660255
2016-05-17 22:37:40,278 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CE336B9482E74877AC135188E045420C
2016-05-17 22:37:40,279 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:40,279 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:40,280 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660111_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524660111_0001_1, initTime=1463524660278
2016-05-17 22:37:40,280 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524660111_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:40,281 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:40,281 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660111_0001_1][Event:DAG_STARTED]: dagID=dag_1463524660111_0001_1, startTime=1463524660280
2016-05-17 22:37:40,281 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:40,281 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524660111_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:40,282 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CE336B9482E74877AC135188E045420C : {C10D93FC12804CB1812ECD0FE01EF311={InputName=C10D93FC12804CB1812ECD0FE01EF311}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:40,282 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C10D93FC12804CB1812ECD0FE01EF311, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:40,282 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C]
2016-05-17 22:37:40,282 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C]
2016-05-17 22:37:40,282 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C]
2016-05-17 22:37:40,282 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, applicationId=application_1463524660111_0001, dagName=[38601E3A8065470E8EB8A9805E879697/ABD7B3287E2543E081B3E632C7B34314] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL
2016-05-17 22:37:40,282 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C]. Starting root input initializers: 1
2016-05-17 22:37:40,283 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:40,283 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C10D93FC12804CB1812ECD0FE01EF311 on vertex vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C]
2016-05-17 22:37:40,286 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:40,286 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C10D93FC12804CB1812ECD0FE01EF311 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:40,287 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:40,288 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:40,289 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:40,289 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C10D93FC12804CB1812ECD0FE01EF311 on vertex vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C]
2016-05-17 22:37:40,290 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:40,290 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C10D93FC12804CB1812ECD0FE01EF311=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:40,290 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C] parallelism set to 1
2016-05-17 22:37:40,290 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:40,290 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660111_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524660111_0001_1_00, eventCount=1
2016-05-17 22:37:40,291 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C]
2016-05-17 22:37:40,291 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D85C93AAC12C46498F228423A9F351C6, vertexId=vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:40,291 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D85C93AAC12C46498F228423A9F351C6, vertexId=vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C]
2016-05-17 22:37:40,292 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D85C93AAC12C46498F228423A9F351C6, vertexName=CE336B9482E74877AC135188E045420C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:40,293 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D85C93AAC12C46498F228423A9F351C6, vertexId=vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C]
2016-05-17 22:37:40,294 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:40,294 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660111_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CE336B9482E74877AC135188E045420C, vertexId=vertex_1463524660111_0001_1_00, initRequestedTime=1463524660282, initedTime=1463524660294, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:40,294 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C] with distanceFromRoot: 0
2016-05-17 22:37:40,294 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:40,295 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CE336B9482E74877AC135188E045420C
2016-05-17 22:37:40,295 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C]
2016-05-17 22:37:40,295 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:40,295 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660111_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524660111_0001_1_00, startRequestedTime=1463524660295, startedTime=1463524660295
2016-05-17 22:37:40,296 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:40,296 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:40,296 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660111_0001_1][Event:TASK_STARTED]: vertexName=CE336B9482E74877AC135188E045420C, taskId=task_1463524660111_0001_1_00_000000, scheduledTime=1463524660296, launchTime=1463524660296
2016-05-17 22:37:40,297 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524660111_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:40,297 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C]
2016-05-17 22:37:40,297 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [38601E3A8065470E8EB8A9805E879697/ABD7B3287E2543E081B3E632C7B34314] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, VertexName: CE336B9482E74877AC135188E045420C, VertexParallelism: 1, TaskAttemptID:attempt_1463524660111_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C10D93FC12804CB1812ECD0FE01EF311, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D85C93AAC12C46498F228423A9F351C6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:40,297 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524660111_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:40,297 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:40,298 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:40,298 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:40,299 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:40,299 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524660111_0001_00_000001
2016-05-17 22:37:40,299 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524660111_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:40,301 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524660111_0001_00_000001
2016-05-17 22:37:40,301 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:40,301 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524660111_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:40,302 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660111_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524660111_0001_00_000001, launchTime=1463524660301
2016-05-17 22:37:40,301 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:40,303 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524660111_0001_00_000001
2016-05-17 22:37:40,304 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524660111_0001_1_00_000000_0] to container: [container_1463524660111_0001_00_000001]
2016-05-17 22:37:40,305 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524660111_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:40,305 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524660111_0001_00_000001 given task: attempt_1463524660111_0001_1_00_000000_0
2016-05-17 22:37:40,305 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524660111_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524660111_0001_1_00_000000_0
2016-05-17 22:37:40,306 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524660111_0001_1_00_000000_0] started. Is using containerId: [container_1463524660111_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:40 Starting to run new task attempt: attempt_1463524660111_0001_1_00_000000_0
2016-05-17 22:37:40,306 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524660111_0001_1_00_000000_0
2016-05-17 22:37:40,307 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:40,307 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:40,307 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:40,307 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:40,307 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:40,307 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:40,308 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [38601E3A8065470E8EB8A9805E879697/ABD7B3287E2543E081B3E632C7B34314] every.every/(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, VertexName: CE336B9482E74877AC135188E045420C, VertexParallelism: 1, TaskAttemptID:attempt_1463524660111_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C10D93FC12804CB1812ECD0FE01EF311, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D85C93AAC12C46498F228423A9F351C6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:40,308 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:40,308 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:40,309 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660111_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CE336B9482E74877AC135188E045420C, taskAttemptId=attempt_1463524660111_0001_1_00_000000_0, startTime=1463524660305, containerId=container_1463524660111_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524660111_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:40,310 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524660111_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:40,311 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524660111_0001, with dag id: dag_1463524660111_0001_1
2016-05-17 22:37:40,310 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524660111_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:40,312 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:40,312 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524660111_0001_1_00_000000_0
2016-05-17 22:37:40,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:40,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:40,312 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524660111_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:40,312 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:40,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:40,315 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D85C93AAC12C46498F228423A9F351C6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:40,316 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:40,316 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:40,316 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D85C93AAC12C46498F228423A9F351C6
2016-05-17 22:37:40,316 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:40,316 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D85C93AAC12C46498F228423A9F351C6
2016-05-17 22:37:40,315 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C10D93FC12804CB1812ECD0FE01EF311, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:40,317 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:40,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C10D93FC12804CB1812ECD0FE01EF311
2016-05-17 22:37:40,318 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C10D93FC12804CB1812ECD0FE01EF311
2016-05-17 22:37:40,323 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:40,324 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C10D93FC12804CB1812ECD0FE01EF311
2016-05-17 22:37:40,324 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C10D93FC12804CB1812ECD0FE01EF311
2016-05-17 22:37:40,330 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CE336B9482E74877AC135188E045420C, ordinal: 0
2016-05-17 22:37:40,331 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE336B9482E74877AC135188E045420C, mem on start (mb), free: 285, total: 385, max: 672
2016-05-17 22:37:40,331 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:40,331 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:40,331 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:40,335 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:40,335 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D85C93AAC12C46498F228423A9F351C6
2016-05-17 22:37:40,335 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:40,335 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:40,336 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:40,336 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:40,336 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:40,336 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:40,337 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:40,337 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:40,337 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D85C93AAC12C46498F228423A9F351C6, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:40,339 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C10D93FC12804CB1812ECD0FE01EF311, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:40,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:40,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C10D93FC12804CB1812ECD0FE01EF311 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:40,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:40,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:40,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C10D93FC12804CB1812ECD0FE01EF311
2016-05-17 22:37:40,340 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:40,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C10D93FC12804CB1812ECD0FE01EF311
2016-05-17 22:37:40,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:40,341 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524660111_0001_1_00_000000_0
2016-05-17 22:37:40,341 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:40,342 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4086c8b4
2016-05-17 22:37:40,343 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4086c8b4
2016-05-17 22:37:40,343 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:40,343 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:40,344 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:40,345 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C10D93FC12804CB1812ECD0FE01EF311
2016-05-17 22:37:40,345 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL"], id: D85C93AAC12C46498F228423A9F351C6
2016-05-17 22:37:40,345 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL"]
2016-05-17 22:37:40,346 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:40,346 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:40,346 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CE336B9482E74877AC135188E045420C, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:40,349 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:40,349 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:40,350 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524660111_0001_1_00_000000_0
2016-05-17 22:37:40,350 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524660111_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:40,351 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524660111_0001_1_00_000000_0
2016-05-17 22:37:40,351 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE336B9482E74877AC135188E045420C, mem on close (mb), free: 284, total: 385, max: 672
2016-05-17 22:37:40 Completed running task attempt: attempt_1463524660111_0001_1_00_000000_0
2016-05-17 22:37:40,352 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524660111_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:40,353 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=792, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524660346, Process_Duration=5, Process_End_Time=1463524660351, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:40,353 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:40,354 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:40,354 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:40,354 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660111_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CE336B9482E74877AC135188E045420C, taskAttemptId=attempt_1463524660111_0001_1_00_000000_0, startTime=1463524660305, finishTime=1463524660353, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=792, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524660346, Process_Duration=5, Process_End_Time=1463524660351, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:40,354 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524660111_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:40,354 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524660111_0001_00_000001
2016-05-17 22:37:40,354 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524660111_0001_1_00_000000_0
2016-05-17 22:37:40,355 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:40,355 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:40,356 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660111_0001_1][Event:TASK_FINISHED]: vertexName=CE336B9482E74877AC135188E045420C, taskId=task_1463524660111_0001_1_00_000000, startTime=1463524660305, finishTime=1463524660355, timeTaken=50, status=SUCCEEDED, successfulAttemptID=attempt_1463524660111_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=792, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524660346, Process_Duration=5, Process_End_Time=1463524660351, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:40,356 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524660111_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:40,356 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C] : 1
2016-05-17 22:37:40,356 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C]
2016-05-17 22:37:40,357 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:40,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660111_0001_1][Event:VERTEX_FINISHED]: vertexName=CE336B9482E74877AC135188E045420C, vertexId=vertex_1463524660111_0001_1_00, initRequestedTime=1463524660282, initedTime=1463524660294, startRequestedTime=1463524660295, startedTime=1463524660295, finishTime=1463524660356, timeTaken=61, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=792, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524660346, Process_Duration=5, Process_End_Time=1463524660351, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524660305, firstTasksToStart=[ task_1463524660111_0001_1_00_000000 ], lastTaskFinishTime=1463524660353, lastTasksToFinish=[ task_1463524660111_0001_1_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524660111_0001_1_00_000000 ], longestDurationTasks=[ task_1463524660111_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:40,358 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:40,358 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524660111_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:40,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660111_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524660111_0001_00_000001, stoppedTime=1463524660358, exitStatus=0
2016-05-17 22:37:40,359 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524660111_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:40,359 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:40,359 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524660111_0001_1
2016-05-17 22:37:40,359 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:40,360 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660111_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524660111_0001_1
2016-05-17 22:37:40,360 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D85C93AAC12C46498F228423A9F351C6 for vertex: vertex_1463524660111_0001_1_00 [CE336B9482E74877AC135188E045420C]
2016-05-17 22:37:40,367 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524660111_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:40,367 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524660111_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:40,367 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524660111_0001_00_000001, Exiting
2016-05-17 22:37:40,367 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524660111_0001_00_000001
2016-05-17 22:37:40,367 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524660111_0001_00_000001
2016-05-17 22:37:40,371 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524660111_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:40,371 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524660111_0001_00_000001 completed successfully
2016-05-17 22:37:40,372 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524660111_0001_1, queueSize=0
2016-05-17 22:37:40,373 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660111_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524660111_0001_1, startTime=1463524660280, finishTime=1463524660371, timeTaken=91, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=792, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524660346, Process_Duration=5, Process_End_Time=1463524660351, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:40,373 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524660111_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:40,373 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:40,373 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524660111_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:40,373 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524660111_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:40 Completed Dag: dag_1463524660111_0001_1
2016-05-17 22:37:40,374 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524660111_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:40,374 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524660111_0001_1_post
2016-05-17 22:37:40,374 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:40,374 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:40,374 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524660111_0001_00_000001
2016-05-17 22:37:40,381 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.-1_UNKNOWN_ALL, applicationId=application_1463524660111_0001
2016-05-17 22:37:40,385 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:40,385 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:40,385 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:40,385 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:40,385 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:40,386 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:40,387 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:40,387 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:40,387 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:40,387 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:40,387 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:40,388 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:40,388 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:40,391 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/38601E3A8065470E8EB8A9805E879697/ABD7B3287E2543E081B3E632C7B34314/.tez/application_1463524660111_0001
2016-05-17 22:37:40,392 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_UNKNOWN_ALL/_temporary
2016-05-17 22:37:40,393 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.301
2016-05-17 22:37:40,462 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_UNKNOWN_ALL
2016-05-17 22:37:40,492 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:40,493 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:40,533 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 22:37:40,544 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-05-17 22:37:40,545 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:40,545 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:40,545 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:40,559 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:40,560 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1
2016-05-17 22:37:40,571 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:40,572 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:40,572 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1"]
2016-05-17 22:37:40,572 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:40,572 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:40,572 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:40,573 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-17 22:37:40,574 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:40,586 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:40,590 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:40,591 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:40,591 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:40,600 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4994CAF4F094B00A566239D6E1CF628/A952464159654A7D9C8A818653A66DBC/.tez/application_1463524660590_0001 doesn't exist and is created
2016-05-17 22:37:40,617 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:40,617 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:40,618 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4994CAF4F094B00A566239D6E1CF628/A952464159654A7D9C8A818653A66DBC/.tez/application_1463524660590_0001
2016-05-17 22:37:40,622 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524660590_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:40,634 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:40,635 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:40,637 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:40,637 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:40,638 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4994CAF4F094B00A566239D6E1CF628/A952464159654A7D9C8A818653A66DBC/.tez/application_1463524660590_0001/localmode-log-dir
2016-05-17 22:37:40,638 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4994CAF4F094B00A566239D6E1CF628/A952464159654A7D9C8A818653A66DBC/.tez/application_1463524660590_0001/localmode-log-dir/history.txt.appattempt_1463524660590_0001_000000, maxErrors=10
2016-05-17 22:37:40,639 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:40,639 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524660590_0001_000000, appSubmitTime=1463524660622, launchTime=1463524660622
2016-05-17 22:37:40,643 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:40,644 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:40,645 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:40,647 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:42061
2016-05-17 22:37:40,651 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:40,652 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:40,655 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524660590_0001_000000, startTime=1463524660655
2016-05-17 22:37:40,655 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:40,655 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:40,717 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:40,717 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:40,718 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.-1_VALUES_-1, applicationId=application_1463524660590_0001, dagName=[E4994CAF4F094B00A566239D6E1CF628/A952464159654A7D9C8A818653A66DBC] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-17 22:37:40,734 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [E4994CAF4F094B00A566239D6E1CF628/A952464159654A7D9C8A818653A66DBC] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-17 22:37:40,735 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524660590_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4994CAF4F094B00A566239D6E1CF628/A952464159654A7D9C8A818653A66DBC/.tez/application_1463524660590_0001/localmode-log-dir/dag_1463524660590_0001_1.dot
2016-05-17 22:37:40,736 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4994CAF4F094B00A566239D6E1CF628/A952464159654A7D9C8A818653A66DBC/.tez/application_1463524660590_0001/localmode-log-dir/dag_1463524660590_0001_1-tez-dag.pb.txt
2016-05-17 22:37:40,742 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524660590_0001_1
2016-05-17 22:37:40,742 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:40,742 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [E4994CAF4F094B00A566239D6E1CF628/A952464159654A7D9C8A818653A66DBC] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-17 22:37:40 Running Dag: dag_1463524660590_0001_1
2016-05-17 22:37:40,755 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660590_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524660590_0001_1, submitTime=1463524660734
2016-05-17 22:37:40,755 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:40,755 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex AE22322E93EA4CF98BB029D17AB4C97A
2016-05-17 22:37:40,756 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:40,756 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:40,758 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660590_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524660590_0001_1, initTime=1463524660755
2016-05-17 22:37:40,758 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524660590_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:40,759 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660590_0001_1][Event:DAG_STARTED]: dagID=dag_1463524660590_0001_1, startTime=1463524660759
2016-05-17 22:37:40,760 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:40,759 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:40,760 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524660590_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:40,760 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AE22322E93EA4CF98BB029D17AB4C97A : {1A5CBE361905426C86221A9409010922={InputName=1A5CBE361905426C86221A9409010922}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:40,760 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 1A5CBE361905426C86221A9409010922, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:40,760 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A]
2016-05-17 22:37:40,761 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A]
2016-05-17 22:37:40,761 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A]
2016-05-17 22:37:40,761 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A]. Starting root input initializers: 1
2016-05-17 22:37:40,760 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.-1_VALUES_-1, applicationId=application_1463524660590_0001, dagName=[E4994CAF4F094B00A566239D6E1CF628/A952464159654A7D9C8A818653A66DBC] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-17 22:37:40,766 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:40,766 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 1A5CBE361905426C86221A9409010922 on vertex vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A]
2016-05-17 22:37:40,768 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:40,769 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 1A5CBE361905426C86221A9409010922 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:40,769 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:40,770 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:40,771 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:40,772 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 1A5CBE361905426C86221A9409010922 on vertex vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A]
2016-05-17 22:37:40,772 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:40,772 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {1A5CBE361905426C86221A9409010922=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:40,772 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A] parallelism set to 1
2016-05-17 22:37:40,772 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:40,773 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660590_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524660590_0001_1_00, eventCount=1
2016-05-17 22:37:40,773 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A]
2016-05-17 22:37:40,773 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=DD66892026BC4F07B4A9748DD2DEED83, vertexId=vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:40,773 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=DD66892026BC4F07B4A9748DD2DEED83, vertexId=vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A]
2016-05-17 22:37:40,774 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=DD66892026BC4F07B4A9748DD2DEED83, vertexName=AE22322E93EA4CF98BB029D17AB4C97A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:40,775 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=DD66892026BC4F07B4A9748DD2DEED83, vertexId=vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A]
2016-05-17 22:37:40,776 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660590_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AE22322E93EA4CF98BB029D17AB4C97A, vertexId=vertex_1463524660590_0001_1_00, initRequestedTime=1463524660760, initedTime=1463524660776, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:40,776 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:40,776 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A] with distanceFromRoot: 0
2016-05-17 22:37:40,776 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:40,776 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AE22322E93EA4CF98BB029D17AB4C97A
2016-05-17 22:37:40,776 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A]
2016-05-17 22:37:40,777 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:40,777 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660590_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524660590_0001_1_00, startRequestedTime=1463524660776, startedTime=1463524660776
2016-05-17 22:37:40,777 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:40,777 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:40,778 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660590_0001_1][Event:TASK_STARTED]: vertexName=AE22322E93EA4CF98BB029D17AB4C97A, taskId=task_1463524660590_0001_1_00_000000, scheduledTime=1463524660777, launchTime=1463524660777
2016-05-17 22:37:40,778 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524660590_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:40,778 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A]
2016-05-17 22:37:40,778 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E4994CAF4F094B00A566239D6E1CF628/A952464159654A7D9C8A818653A66DBC] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1, VertexName: AE22322E93EA4CF98BB029D17AB4C97A, VertexParallelism: 1, TaskAttemptID:attempt_1463524660590_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1A5CBE361905426C86221A9409010922, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DD66892026BC4F07B4A9748DD2DEED83, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:40,778 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524660590_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:40,778 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:40,779 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:40,779 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:40,780 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:40,780 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524660590_0001_00_000001
2016-05-17 22:37:40,780 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524660590_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:40,782 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524660590_0001, with dag id: dag_1463524660590_0001_1
2016-05-17 22:37:40,782 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524660590_0001_00_000001
2016-05-17 22:37:40,782 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:40,783 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524660590_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:40,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:40,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660590_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524660590_0001_00_000001, launchTime=1463524660783
2016-05-17 22:37:40,786 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524660590_0001_00_000001
2016-05-17 22:37:40,787 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524660590_0001_1_00_000000_0] to container: [container_1463524660590_0001_00_000001]
2016-05-17 22:37:40,787 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524660590_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:40,787 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524660590_0001_00_000001 given task: attempt_1463524660590_0001_1_00_000000_0
2016-05-17 22:37:40,788 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524660590_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524660590_0001_1_00_000000_0
2016-05-17 22:37:40,788 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524660590_0001_1_00_000000_0] started. Is using containerId: [container_1463524660590_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:40,789 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660590_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AE22322E93EA4CF98BB029D17AB4C97A, taskAttemptId=attempt_1463524660590_0001_1_00_000000_0, startTime=1463524660788, containerId=container_1463524660590_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524660590_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:40,789 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524660590_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:40,789 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524660590_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:40,789 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:40 Starting to run new task attempt: attempt_1463524660590_0001_1_00_000000_0
2016-05-17 22:37:40,790 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524660590_0001_1_00_000000_0
2016-05-17 22:37:40,791 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:40,791 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:40,791 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:40,791 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:40,792 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:40,792 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:40,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E4994CAF4F094B00A566239D6E1CF628/A952464159654A7D9C8A818653A66DBC] every.every/(1/1) ...eld'_RESULTS.-1_VALUES_-1, VertexName: AE22322E93EA4CF98BB029D17AB4C97A, VertexParallelism: 1, TaskAttemptID:attempt_1463524660590_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1A5CBE361905426C86221A9409010922, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DD66892026BC4F07B4A9748DD2DEED83, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:40,792 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:40,793 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524660590_0001_1_00_000000_0
2016-05-17 22:37:40,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:40,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:40,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:40,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=1A5CBE361905426C86221A9409010922, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:40,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:40,796 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DD66892026BC4F07B4A9748DD2DEED83, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:40,796 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:40,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 1A5CBE361905426C86221A9409010922
2016-05-17 22:37:40,801 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 1A5CBE361905426C86221A9409010922
2016-05-17 22:37:40,802 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524660590_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:40,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:40,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DD66892026BC4F07B4A9748DD2DEED83
2016-05-17 22:37:40,805 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:40,805 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: DD66892026BC4F07B4A9748DD2DEED83
2016-05-17 22:37:40,805 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:40,806 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 1A5CBE361905426C86221A9409010922
2016-05-17 22:37:40,806 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 1A5CBE361905426C86221A9409010922
2016-05-17 22:37:40,818 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AE22322E93EA4CF98BB029D17AB4C97A, ordinal: 0
2016-05-17 22:37:40,818 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AE22322E93EA4CF98BB029D17AB4C97A, mem on start (mb), free: 208, total: 385, max: 672
2016-05-17 22:37:40,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:40,818 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:40,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DD66892026BC4F07B4A9748DD2DEED83
2016-05-17 22:37:40,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:40,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:40,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:40,819 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:40,820 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:40,820 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:40,820 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:40,820 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:40,820 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:40,821 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:40,821 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 1A5CBE361905426C86221A9409010922, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:40,822 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DD66892026BC4F07B4A9748DD2DEED83, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:40,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:40,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 1A5CBE361905426C86221A9409010922
2016-05-17 22:37:40,822 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:40,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 1A5CBE361905426C86221A9409010922
2016-05-17 22:37:40,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 1A5CBE361905426C86221A9409010922 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:40,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:40,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:40,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:40,824 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524660590_0001_1_00_000000_0
2016-05-17 22:37:40,826 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:40,827 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:40,827 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@668ded54
2016-05-17 22:37:40,828 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 1A5CBE361905426C86221A9409010922
2016-05-17 22:37:40,828 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1"], id: DD66892026BC4F07B4A9748DD2DEED83
2016-05-17 22:37:40,828 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1"]
2016-05-17 22:37:40,828 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:40,828 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:40,828 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@668ded54
2016-05-17 22:37:40,829 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:40,829 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:40,830 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AE22322E93EA4CF98BB029D17AB4C97A, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:40,831 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:40,833 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:40,834 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:40,834 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:40,834 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:40,835 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524660590_0001_1_00_000000_0
2016-05-17 22:37:40,835 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524660590_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:40,836 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524660590_0001_1_00_000000_0
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:40,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524660830, Process_Duration=6, Process_End_Time=1463524660836, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:37:40,842 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AE22322E93EA4CF98BB029D17AB4C97A, mem on close (mb), free: 205, total: 385, max: 672
2016-05-17 22:37:40,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=AE22322E93EA4CF98BB029D17AB4C97A, index=0
2016-05-17 22:37:40,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=AE22322E93EA4CF98BB029D17AB4C97A, sourceVertex=1A5CBE361905426C86221A9409010922
2016-05-17 22:37:40,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=AE22322E93EA4CF98BB029D17AB4C97A, sourceVertex=DD66892026BC4F07B4A9748DD2DEED83
2016-05-17 22:37:40,843 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524660590_0001_00_000001
2016-05-17 22:37:40,843 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524660590_0001_00_000001
2016-05-17 22:37:40,845 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:40,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660590_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AE22322E93EA4CF98BB029D17AB4C97A, taskAttemptId=attempt_1463524660590_0001_1_00_000000_0, startTime=1463524660788, finishTime=1463524660842, timeTaken=54, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524660830, Process_Duration=6, Process_End_Time=1463524660836, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:37:40,846 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524660590_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:37:40,846 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524660590_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:37:40,846 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524660590_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:37:40,847 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660590_0001_1][Event:TASK_FINISHED]: vertexName=AE22322E93EA4CF98BB029D17AB4C97A, taskId=task_1463524660590_0001_1_00_000000, startTime=1463524660788, finishTime=1463524660847, timeTaken=59, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
], counters=Counters: 0
2016-05-17 22:37:40,847 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524660590_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:37:40,847 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:40,847 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:40,847 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524660590_0001_00_000001 completed but with errors
2016-05-17 22:37:40,847 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A] : 1
2016-05-17 22:37:40,851 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A] because task failed: task_1463524660590_0001_1_00_000000
2016-05-17 22:37:40,852 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:37:40,852 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:37:40,852 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A]
2016-05-17 22:37:40,853 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=DD66892026BC4F07B4A9748DD2DEED83, vertexId=vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A]
2016-05-17 22:37:40,857 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:40,858 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660590_0001_1][Event:VERTEX_FINISHED]: vertexName=AE22322E93EA4CF98BB029D17AB4C97A, vertexId=vertex_1463524660590_0001_1_00, initRequestedTime=1463524660760, initedTime=1463524660776, startRequestedTime=1463524660776, startedTime=1463524660776, finishTime=1463524660857, timeTaken=81, status=FAILED, diagnostics=Task failed, taskId=task_1463524660590_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524660788, firstTasksToStart=[ task_1463524660590_0001_1_00_000000 ], lastTaskFinishTime=1463524660842, lastTasksToFinish=[ task_1463524660590_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:37:40,858 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:37:40,859 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:40,859 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660590_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524660590_0001_00_000001, stoppedTime=1463524660859, exitStatus=0
2016-05-17 22:37:40,859 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524660590_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:40,859 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524660590_0001_1_00_000000_0 failed: true container: container_1463524660590_0001_00_000001 numFailedTAs: 0
2016-05-17 22:37:40,861 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:37:40,861 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:37:40,861 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524660590_0001_1
2016-05-17 22:37:40,861 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: DD66892026BC4F07B4A9748DD2DEED83 for vertex: vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A]
2016-05-17 22:37:40,863 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524660590_0001_1, queueSize=0
2016-05-17 22:37:40,863 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:40,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524660590_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524660590_0001_1, startTime=1463524660759, finishTime=1463524660863, timeTaken=104, status=FAILED, diagnostics=Vertex failed, vertexName=AE22322E93EA4CF98BB029D17AB4C97A, vertexId=vertex_1463524660590_0001_1_00, diagnostics=[Task failed, taskId=task_1463524660590_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524660590_0001_1_00 [AE22322E93EA4CF98BB029D17AB4C97A] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:37:40,865 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524660590_0001_1 finished with state: FAILED
2016-05-17 22:37:40,865 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524660590_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:40,866 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524660590_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:37:40,868 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524660590_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:40,868 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524660590_0001_00_000001
2016-05-17 22:37:40 Completed Dag: dag_1463524660590_0001_1
2016-05-17 22:37:40,868 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524660590_0001_1, dagState=FAILED
2016-05-17 22:37:40,868 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524660590_0001_1_post
2016-05-17 22:37:40,868 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:40,868 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:40,869 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524660590_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:37:40,875 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...eld'_RESULTS.-1_VALUES_-1, applicationId=application_1463524660590_0001
2016-05-17 22:37:40,878 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:40,878 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:40,879 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:40,879 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:40,879 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:40,879 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:40,880 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:40,880 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:40,880 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:40,881 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:40,881 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:40,883 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:40,883 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:40,887 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/E4994CAF4F094B00A566239D6E1CF628/A952464159654A7D9C8A818653A66DBC/.tez/application_1463524660590_0001
2016-05-17 22:37:40,887 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-17 22:37:40,888 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...eld'_RESULTS.-1_VALUES_-1
2016-05-17 22:37:40,889 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-17 22:37:40,889 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_-1/_temporary
2016-05-17 22:37:40,892 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.318
2016-05-17 22:37:40,892 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_-1
2016-05-17 22:37:40,915 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:40,916 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:40,945 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-17 22:37:40,949 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.008
2016-05-17 22:37:40,950 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:40,950 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:40,951 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_'field2'
2016-05-17 22:37:40,974 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:40,974 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:41,010 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 22:37:41,018 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-17 22:37:41,019 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:41,019 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:41,020 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:41,034 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:41,035 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS
2016-05-17 22:37:41,046 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:41,046 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:41,046 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS"]
2016-05-17 22:37:41,047 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:41,047 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:41,047 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:41,048 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-17 22:37:41,048 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:41,067 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:41,072 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:41,072 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:41,072 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:41,083 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A601362D80A0447090EAF705329FFAAF/D59133CC8C724B8B845AE1EBACC52299/.tez/application_1463524661072_0001 doesn't exist and is created
2016-05-17 22:37:41,101 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:41,101 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:41,102 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A601362D80A0447090EAF705329FFAAF/D59133CC8C724B8B845AE1EBACC52299/.tez/application_1463524661072_0001
2016-05-17 22:37:41,107 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524661072_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:41,117 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:41,118 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:41,120 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:41,120 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:41,121 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A601362D80A0447090EAF705329FFAAF/D59133CC8C724B8B845AE1EBACC52299/.tez/application_1463524661072_0001/localmode-log-dir
2016-05-17 22:37:41,121 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A601362D80A0447090EAF705329FFAAF/D59133CC8C724B8B845AE1EBACC52299/.tez/application_1463524661072_0001/localmode-log-dir/history.txt.appattempt_1463524661072_0001_000000, maxErrors=10
2016-05-17 22:37:41,121 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:41,121 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524661072_0001_000000, appSubmitTime=1463524661105, launchTime=1463524661105
2016-05-17 22:37:41,127 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:41,128 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56240
2016-05-17 22:37:41,128 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:41,130 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:41,130 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:41,130 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:41,133 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524661072_0001_000000, startTime=1463524661133
2016-05-17 22:37:41,133 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:41,133 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:41,137 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:41,137 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:41,202 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:41,202 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:41,202 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...RESULTS.-1_VALUES_RESULTS, applicationId=application_1463524661072_0001, dagName=[A601362D80A0447090EAF705329FFAAF/D59133CC8C724B8B845AE1EBACC52299] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-17 22:37:41,237 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A601362D80A0447090EAF705329FFAAF/D59133CC8C724B8B845AE1EBACC52299] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-17 22:37:41,239 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524661072_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A601362D80A0447090EAF705329FFAAF/D59133CC8C724B8B845AE1EBACC52299/.tez/application_1463524661072_0001/localmode-log-dir/dag_1463524661072_0001_1.dot
2016-05-17 22:37:41,239 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A601362D80A0447090EAF705329FFAAF/D59133CC8C724B8B845AE1EBACC52299/.tez/application_1463524661072_0001/localmode-log-dir/dag_1463524661072_0001_1-tez-dag.pb.txt
2016-05-17 22:37:41,245 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524661072_0001_1
2016-05-17 22:37:41,245 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:41,246 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A601362D80A0447090EAF705329FFAAF/D59133CC8C724B8B845AE1EBACC52299] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-17 22:37:41 Running Dag: dag_1463524661072_0001_1
2016-05-17 22:37:41,260 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661072_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524661072_0001_1, submitTime=1463524661237
2016-05-17 22:37:41,261 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BFCC0DBFA0CF4529A4ADB4F882B01B0F
2016-05-17 22:37:41,261 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:41,261 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:41,261 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661072_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524661072_0001_1, initTime=1463524661260
2016-05-17 22:37:41,262 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524661072_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:41,262 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:41,262 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661072_0001_1][Event:DAG_STARTED]: dagID=dag_1463524661072_0001_1, startTime=1463524661262
2016-05-17 22:37:41,263 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:41,263 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524661072_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:41,263 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BFCC0DBFA0CF4529A4ADB4F882B01B0F : {045DE124AA964C7980000C7A57298D5D={InputName=045DE124AA964C7980000C7A57298D5D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:41,264 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 045DE124AA964C7980000C7A57298D5D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:41,264 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F]
2016-05-17 22:37:41,264 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F]
2016-05-17 22:37:41,264 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F]
2016-05-17 22:37:41,262 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:41,265 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F]. Starting root input initializers: 1
2016-05-17 22:37:41,265 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:41,266 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 045DE124AA964C7980000C7A57298D5D on vertex vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F]
2016-05-17 22:37:41,268 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:41,268 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 045DE124AA964C7980000C7A57298D5D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:41,271 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:41,271 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:41,272 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...RESULTS.-1_VALUES_RESULTS, applicationId=application_1463524661072_0001, dagName=[A601362D80A0447090EAF705329FFAAF/D59133CC8C724B8B845AE1EBACC52299] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-17 22:37:41,273 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:41,278 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 045DE124AA964C7980000C7A57298D5D on vertex vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F]
2016-05-17 22:37:41,278 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:41,278 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {045DE124AA964C7980000C7A57298D5D=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:41,279 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F] parallelism set to 1
2016-05-17 22:37:41,279 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:41,279 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661072_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524661072_0001_1_00, eventCount=1
2016-05-17 22:37:41,279 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F]
2016-05-17 22:37:41,279 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=54D5ED6D23964B188583E41976A85E33, vertexId=vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:41,280 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=54D5ED6D23964B188583E41976A85E33, vertexId=vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F]
2016-05-17 22:37:41,282 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=54D5ED6D23964B188583E41976A85E33, vertexName=BFCC0DBFA0CF4529A4ADB4F882B01B0F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:41,282 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=54D5ED6D23964B188583E41976A85E33, vertexId=vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F]
2016-05-17 22:37:41,285 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:41,285 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661072_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BFCC0DBFA0CF4529A4ADB4F882B01B0F, vertexId=vertex_1463524661072_0001_1_00, initRequestedTime=1463524661263, initedTime=1463524661284, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:41,286 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F] with distanceFromRoot: 0
2016-05-17 22:37:41,286 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:41,287 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BFCC0DBFA0CF4529A4ADB4F882B01B0F
2016-05-17 22:37:41,287 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F]
2016-05-17 22:37:41,287 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661072_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524661072_0001_1_00, startRequestedTime=1463524661287, startedTime=1463524661287
2016-05-17 22:37:41,287 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:41,288 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:41,289 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:41,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661072_0001_1][Event:TASK_STARTED]: vertexName=BFCC0DBFA0CF4529A4ADB4F882B01B0F, taskId=task_1463524661072_0001_1_00_000000, scheduledTime=1463524661289, launchTime=1463524661289
2016-05-17 22:37:41,290 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524661072_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:41,290 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F]
2016-05-17 22:37:41,291 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A601362D80A0447090EAF705329FFAAF/D59133CC8C724B8B845AE1EBACC52299] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS, VertexName: BFCC0DBFA0CF4529A4ADB4F882B01B0F, VertexParallelism: 1, TaskAttemptID:attempt_1463524661072_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=045DE124AA964C7980000C7A57298D5D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=54D5ED6D23964B188583E41976A85E33, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:41,291 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524661072_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:41,291 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:41,295 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:41,296 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:41,296 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:41,296 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524661072_0001_00_000001
2016-05-17 22:37:41,297 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524661072_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:41,298 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524661072_0001_00_000001
2016-05-17 22:37:41,299 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:41,300 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524661072_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:41,300 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:41,300 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661072_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524661072_0001_00_000001, launchTime=1463524661300
2016-05-17 22:37:41,301 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524661072_0001_00_000001
2016-05-17 22:37:41,302 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524661072_0001_1_00_000000_0] to container: [container_1463524661072_0001_00_000001]
2016-05-17 22:37:41,302 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524661072_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:41,302 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524661072_0001, with dag id: dag_1463524661072_0001_1
2016-05-17 22:37:41,305 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524661072_0001_00_000001 given task: attempt_1463524661072_0001_1_00_000000_0
2016-05-17 22:37:41,305 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524661072_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524661072_0001_1_00_000000_0
2016-05-17 22:37:41,305 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524661072_0001_1_00_000000_0] started. Is using containerId: [container_1463524661072_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:41,305 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:41,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661072_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BFCC0DBFA0CF4529A4ADB4F882B01B0F, taskAttemptId=attempt_1463524661072_0001_1_00_000000_0, startTime=1463524661303, containerId=container_1463524661072_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524661072_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:41,306 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524661072_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:41,306 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524661072_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:41 Starting to run new task attempt: attempt_1463524661072_0001_1_00_000000_0
2016-05-17 22:37:41,307 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524661072_0001_1_00_000000_0
2016-05-17 22:37:41,307 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:41,307 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:41,307 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:41,307 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:41,307 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:41,308 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:41,308 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A601362D80A0447090EAF705329FFAAF/D59133CC8C724B8B845AE1EBACC52299] every.every/(1/1) ...RESULTS.-1_VALUES_RESULTS, VertexName: BFCC0DBFA0CF4529A4ADB4F882B01B0F, VertexParallelism: 1, TaskAttemptID:attempt_1463524661072_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=045DE124AA964C7980000C7A57298D5D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=54D5ED6D23964B188583E41976A85E33, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:41,308 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:41,311 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524661072_0001_1_00_000000_0
2016-05-17 22:37:41,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:41,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:41,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:41,316 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:41,317 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=045DE124AA964C7980000C7A57298D5D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:41,321 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:41,321 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 045DE124AA964C7980000C7A57298D5D
2016-05-17 22:37:41,321 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 045DE124AA964C7980000C7A57298D5D
2016-05-17 22:37:41,320 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524661072_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:41,320 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=54D5ED6D23964B188583E41976A85E33, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:41,324 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:41,324 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 54D5ED6D23964B188583E41976A85E33
2016-05-17 22:37:41,324 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:41,325 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 54D5ED6D23964B188583E41976A85E33
2016-05-17 22:37:41,327 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:41,327 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 045DE124AA964C7980000C7A57298D5D
2016-05-17 22:37:41,328 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 045DE124AA964C7980000C7A57298D5D
2016-05-17 22:37:41,331 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BFCC0DBFA0CF4529A4ADB4F882B01B0F, ordinal: 0
2016-05-17 22:37:41,332 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BFCC0DBFA0CF4529A4ADB4F882B01B0F, mem on start (mb), free: 336, total: 386, max: 672
2016-05-17 22:37:41,332 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:41,332 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:41,332 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:41,340 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:41,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 54D5ED6D23964B188583E41976A85E33
2016-05-17 22:37:41,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:41,341 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:41,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:41,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:41,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:41,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:41,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:41,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:41,341 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 045DE124AA964C7980000C7A57298D5D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:41,341 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 54D5ED6D23964B188583E41976A85E33, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:41,341 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:41,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 045DE124AA964C7980000C7A57298D5D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:41,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 045DE124AA964C7980000C7A57298D5D
2016-05-17 22:37:41,342 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:41,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 045DE124AA964C7980000C7A57298D5D
2016-05-17 22:37:41,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:41,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:41,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:41,343 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524661072_0001_1_00_000000_0
2016-05-17 22:37:41,343 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:41,345 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@124ca3bd
2016-05-17 22:37:41,347 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:41,349 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 045DE124AA964C7980000C7A57298D5D
2016-05-17 22:37:41,349 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS"], id: 54D5ED6D23964B188583E41976A85E33
2016-05-17 22:37:41,349 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS"]
2016-05-17 22:37:41,350 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:41,350 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:41,348 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@124ca3bd
2016-05-17 22:37:41,350 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:41,351 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:41,352 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BFCC0DBFA0CF4529A4ADB4F882B01B0F, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:41,352 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:41,356 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:41,359 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:41,363 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:41,364 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:41,365 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524661072_0001_1_00_000000_0
2016-05-17 22:37:41,366 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524661072_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:41,368 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524661072_0001_1_00_000000_0
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:41,374 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524661352, Process_Duration=16, Process_End_Time=1463524661368, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:37:41,374 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:41,374 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661072_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BFCC0DBFA0CF4529A4ADB4F882B01B0F, taskAttemptId=attempt_1463524661072_0001_1_00_000000_0, startTime=1463524661303, finishTime=1463524661374, timeTaken=71, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524661352, Process_Duration=16, Process_End_Time=1463524661368, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:37:41,374 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524661072_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:37:41,375 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524661072_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:37:41,375 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661072_0001_1][Event:TASK_FINISHED]: vertexName=BFCC0DBFA0CF4529A4ADB4F882B01B0F, taskId=task_1463524661072_0001_1_00_000000, startTime=1463524661303, finishTime=1463524661375, timeTaken=72, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
], counters=Counters: 0
2016-05-17 22:37:41,375 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524661072_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:37:41,375 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:41,376 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:41,376 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F] : 1
2016-05-17 22:37:41,376 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F] because task failed: task_1463524661072_0001_1_00_000000
2016-05-17 22:37:41,376 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:37:41,376 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:37:41,376 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F]
2016-05-17 22:37:41,376 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=54D5ED6D23964B188583E41976A85E33, vertexId=vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F]
2016-05-17 22:37:41,379 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BFCC0DBFA0CF4529A4ADB4F882B01B0F, mem on close (mb), free: 333, total: 386, max: 672
2016-05-17 22:37:41,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=BFCC0DBFA0CF4529A4ADB4F882B01B0F, index=0
2016-05-17 22:37:41,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=BFCC0DBFA0CF4529A4ADB4F882B01B0F, sourceVertex=045DE124AA964C7980000C7A57298D5D
2016-05-17 22:37:41,379 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:41,380 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661072_0001_1][Event:VERTEX_FINISHED]: vertexName=BFCC0DBFA0CF4529A4ADB4F882B01B0F, vertexId=vertex_1463524661072_0001_1_00, initRequestedTime=1463524661263, initedTime=1463524661284, startRequestedTime=1463524661287, startedTime=1463524661287, finishTime=1463524661378, timeTaken=91, status=FAILED, diagnostics=Task failed, taskId=task_1463524661072_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524661303, firstTasksToStart=[ task_1463524661072_0001_1_00_000000 ], lastTaskFinishTime=1463524661374, lastTasksToFinish=[ task_1463524661072_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:37:41,380 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=BFCC0DBFA0CF4529A4ADB4F882B01B0F, sourceVertex=54D5ED6D23964B188583E41976A85E33
2016-05-17 22:37:41,380 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:37:41,380 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524661072_0001_00_000001
2016-05-17 22:37:41,380 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524661072_0001_00_000001
2016-05-17 22:37:41,380 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:37:41,381 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:37:41,381 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524661072_0001_1
2016-05-17 22:37:41,383 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 54D5ED6D23964B188583E41976A85E33 for vertex: vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F]
2016-05-17 22:37:41,384 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524661072_0001_1, queueSize=0
2016-05-17 22:37:41,384 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661072_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524661072_0001_1, startTime=1463524661262, finishTime=1463524661383, timeTaken=121, status=FAILED, diagnostics=Vertex failed, vertexName=BFCC0DBFA0CF4529A4ADB4F882B01B0F, vertexId=vertex_1463524661072_0001_1_00, diagnostics=[Task failed, taskId=task_1463524661072_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524661072_0001_1_00 [BFCC0DBFA0CF4529A4ADB4F882B01B0F] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:37:41,384 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524661072_0001_1 finished with state: FAILED
2016-05-17 22:37:41,384 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524661072_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:41,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661072_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524661072_0001_00_000001, stoppedTime=1463524661385, exitStatus=0
2016-05-17 22:37:41,385 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524661072_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:41,385 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524661072_0001_1_00_000000_0 failed: true container: container_1463524661072_0001_00_000001 numFailedTAs: 0
2016-05-17 22:37:41 Completed Dag: dag_1463524661072_0001_1
2016-05-17 22:37:41,387 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:41,388 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524661072_0001_1, dagState=FAILED
2016-05-17 22:37:41,388 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524661072_0001_1_post
2016-05-17 22:37:41,388 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:41,388 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:41,388 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:41,387 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524661072_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:37:41,388 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524661072_0001_00_000001
2016-05-17 22:37:41,389 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524661072_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:37:41,389 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524661072_0001_00_000001 completed but with errors
2016-05-17 22:37:41,389 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524661072_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:37:41,389 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524661072_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:37:41,389 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524661072_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:37:41,395 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...RESULTS.-1_VALUES_RESULTS, applicationId=application_1463524661072_0001
2016-05-17 22:37:41,411 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:41,411 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:41,412 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:41,412 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:41,412 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:41,412 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:41,413 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:41,413 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:41,413 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:41,413 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:41,414 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:41,414 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:41,414 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:41,418 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/A601362D80A0447090EAF705329FFAAF/D59133CC8C724B8B845AE1EBACC52299/.tez/application_1463524661072_0001
2016-05-17 22:37:41,419 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-17 22:37:41,419 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...RESULTS.-1_VALUES_RESULTS
2016-05-17 22:37:41,420 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-17 22:37:41,420 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_RESULTS/_temporary
2016-05-17 22:37:41,420 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.374
2016-05-17 22:37:41,420 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_RESULTS
2016-05-17 22:37:41,444 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:41,444 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:41,481 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 22:37:41,492 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-17 22:37:41,492 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:41,492 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:41,494 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:41,509 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:41,509 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL
2016-05-17 22:37:41,521 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:41,522 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:41,522 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL"]
2016-05-17 22:37:41,522 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:41,522 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:41,522 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:41,523 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-17 22:37:41,524 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:41,537 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:41,541 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:41,542 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:41,542 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:41,552 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/01CCC59F717C4FFA90761E41CF924967/C2B7D16053794330AD340F4478EEEF04/.tez/application_1463524661542_0001 doesn't exist and is created
2016-05-17 22:37:41,570 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:41,570 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:41,570 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/01CCC59F717C4FFA90761E41CF924967/C2B7D16053794330AD340F4478EEEF04/.tez/application_1463524661542_0001
2016-05-17 22:37:41,576 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524661542_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:41,592 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:41,593 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:41,596 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:41,596 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:41,597 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/01CCC59F717C4FFA90761E41CF924967/C2B7D16053794330AD340F4478EEEF04/.tez/application_1463524661542_0001/localmode-log-dir
2016-05-17 22:37:41,598 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/01CCC59F717C4FFA90761E41CF924967/C2B7D16053794330AD340F4478EEEF04/.tez/application_1463524661542_0001/localmode-log-dir/history.txt.appattempt_1463524661542_0001_000000, maxErrors=10
2016-05-17 22:37:41,598 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:41,600 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524661542_0001_000000, appSubmitTime=1463524661575, launchTime=1463524661576
2016-05-17 22:37:41,604 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:41,609 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:41,611 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:60998
2016-05-17 22:37:41,613 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:41,614 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:41,615 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:41,618 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524661542_0001_000000, startTime=1463524661618
2016-05-17 22:37:41,618 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:41,618 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:41,670 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:41,670 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:41,670 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_VALUES_ALL, applicationId=application_1463524661542_0001, dagName=[01CCC59F717C4FFA90761E41CF924967/C2B7D16053794330AD340F4478EEEF04] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-17 22:37:41,686 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [01CCC59F717C4FFA90761E41CF924967/C2B7D16053794330AD340F4478EEEF04] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-17 22:37:41,687 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524661542_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/01CCC59F717C4FFA90761E41CF924967/C2B7D16053794330AD340F4478EEEF04/.tez/application_1463524661542_0001/localmode-log-dir/dag_1463524661542_0001_1.dot
2016-05-17 22:37:41,688 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/01CCC59F717C4FFA90761E41CF924967/C2B7D16053794330AD340F4478EEEF04/.tez/application_1463524661542_0001/localmode-log-dir/dag_1463524661542_0001_1-tez-dag.pb.txt
2016-05-17 22:37:41,694 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524661542_0001_1
2016-05-17 22:37:41,694 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:41,694 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [01CCC59F717C4FFA90761E41CF924967/C2B7D16053794330AD340F4478EEEF04] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-17 22:37:41 Running Dag: dag_1463524661542_0001_1
2016-05-17 22:37:41,706 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:41,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661542_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524661542_0001_1, submitTime=1463524661686
2016-05-17 22:37:41,707 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DECCB84D380A4C5F8B3DD65DCD6BDEC8
2016-05-17 22:37:41,708 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:41,708 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:41,709 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661542_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524661542_0001_1, initTime=1463524661707
2016-05-17 22:37:41,709 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524661542_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:41,710 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.-1_VALUES_ALL, applicationId=application_1463524661542_0001, dagName=[01CCC59F717C4FFA90761E41CF924967/C2B7D16053794330AD340F4478EEEF04] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-17 22:37:41,710 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661542_0001_1][Event:DAG_STARTED]: dagID=dag_1463524661542_0001_1, startTime=1463524661710
2016-05-17 22:37:41,710 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:41,710 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524661542_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:41,710 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DECCB84D380A4C5F8B3DD65DCD6BDEC8 : {4F036F61D8C54C9A821FDD52FBF96A40={InputName=4F036F61D8C54C9A821FDD52FBF96A40}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:41,710 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4F036F61D8C54C9A821FDD52FBF96A40, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:41,711 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8]
2016-05-17 22:37:41,711 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8]
2016-05-17 22:37:41,711 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:41,711 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8]
2016-05-17 22:37:41,711 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8]. Starting root input initializers: 1
2016-05-17 22:37:41,712 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:41,712 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4F036F61D8C54C9A821FDD52FBF96A40 on vertex vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8]
2016-05-17 22:37:41,714 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:41,714 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4F036F61D8C54C9A821FDD52FBF96A40 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:41,716 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:41,716 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:41,718 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:41,719 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4F036F61D8C54C9A821FDD52FBF96A40 on vertex vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8]
2016-05-17 22:37:41,719 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:41,719 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4F036F61D8C54C9A821FDD52FBF96A40=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:41,719 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8] parallelism set to 1
2016-05-17 22:37:41,720 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:41,720 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661542_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524661542_0001_1_00, eventCount=1
2016-05-17 22:37:41,720 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8]
2016-05-17 22:37:41,720 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=AC777E1E57F949C98EAFF057F29CBA66, vertexId=vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:41,720 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=AC777E1E57F949C98EAFF057F29CBA66, vertexId=vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8]
2016-05-17 22:37:41,722 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=AC777E1E57F949C98EAFF057F29CBA66, vertexName=DECCB84D380A4C5F8B3DD65DCD6BDEC8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:41,723 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=AC777E1E57F949C98EAFF057F29CBA66, vertexId=vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8]
2016-05-17 22:37:41,724 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:41,724 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661542_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DECCB84D380A4C5F8B3DD65DCD6BDEC8, vertexId=vertex_1463524661542_0001_1_00, initRequestedTime=1463524661710, initedTime=1463524661724, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:41,725 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8] with distanceFromRoot: 0
2016-05-17 22:37:41,725 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:41,725 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DECCB84D380A4C5F8B3DD65DCD6BDEC8
2016-05-17 22:37:41,726 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8]
2016-05-17 22:37:41,726 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:41,726 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661542_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524661542_0001_1_00, startRequestedTime=1463524661725, startedTime=1463524661725
2016-05-17 22:37:41,727 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:41,727 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:41,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661542_0001_1][Event:TASK_STARTED]: vertexName=DECCB84D380A4C5F8B3DD65DCD6BDEC8, taskId=task_1463524661542_0001_1_00_000000, scheduledTime=1463524661727, launchTime=1463524661727
2016-05-17 22:37:41,728 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524661542_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:41,728 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8]
2016-05-17 22:37:41,728 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [01CCC59F717C4FFA90761E41CF924967/C2B7D16053794330AD340F4478EEEF04] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL, VertexName: DECCB84D380A4C5F8B3DD65DCD6BDEC8, VertexParallelism: 1, TaskAttemptID:attempt_1463524661542_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4F036F61D8C54C9A821FDD52FBF96A40, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AC777E1E57F949C98EAFF057F29CBA66, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:41,728 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524661542_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:41,729 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:41,729 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:41,730 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:41,730 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:41,730 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524661542_0001_00_000001
2016-05-17 22:37:41,730 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524661542_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:41,732 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524661542_0001_00_000001
2016-05-17 22:37:41,732 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:41,733 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524661542_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:41,734 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:41,734 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661542_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524661542_0001_00_000001, launchTime=1463524661733
2016-05-17 22:37:41,735 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524661542_0001_00_000001
2016-05-17 22:37:41,735 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524661542_0001_1_00_000000_0] to container: [container_1463524661542_0001_00_000001]
2016-05-17 22:37:41,736 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524661542_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:41,736 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524661542_0001_1_00_000000_0] started. Is using containerId: [container_1463524661542_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:41,736 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661542_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DECCB84D380A4C5F8B3DD65DCD6BDEC8, taskAttemptId=attempt_1463524661542_0001_1_00_000000_0, startTime=1463524661736, containerId=container_1463524661542_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524661542_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:41,737 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524661542_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:41,737 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524661542_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:41,737 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:41,739 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524661542_0001, with dag id: dag_1463524661542_0001_1
2016-05-17 22:37:41,741 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524661542_0001_00_000001 given task: attempt_1463524661542_0001_1_00_000000_0
2016-05-17 22:37:41,742 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524661542_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524661542_0001_1_00_000000_0
2016-05-17 22:37:41 Starting to run new task attempt: attempt_1463524661542_0001_1_00_000000_0
2016-05-17 22:37:41,743 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524661542_0001_1_00_000000_0
2016-05-17 22:37:41,743 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:41,744 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:41,744 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:41,744 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:41,744 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:41,744 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:41,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [01CCC59F717C4FFA90761E41CF924967/C2B7D16053794330AD340F4478EEEF04] every.every/(1/1) ...ld'_RESULTS.-1_VALUES_ALL, VertexName: DECCB84D380A4C5F8B3DD65DCD6BDEC8, VertexParallelism: 1, TaskAttemptID:attempt_1463524661542_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4F036F61D8C54C9A821FDD52FBF96A40, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AC777E1E57F949C98EAFF057F29CBA66, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:41,744 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:41,745 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524661542_0001_1_00_000000_0
2016-05-17 22:37:41,746 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524661542_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:41,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:41,747 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:41,748 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4F036F61D8C54C9A821FDD52FBF96A40, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:41,748 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:41,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AC777E1E57F949C98EAFF057F29CBA66, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:41,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:41,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AC777E1E57F949C98EAFF057F29CBA66
2016-05-17 22:37:41,751 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:41,751 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: AC777E1E57F949C98EAFF057F29CBA66
2016-05-17 22:37:41,751 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:41,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:41,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4F036F61D8C54C9A821FDD52FBF96A40
2016-05-17 22:37:41,755 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4F036F61D8C54C9A821FDD52FBF96A40
2016-05-17 22:37:41,760 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:41,761 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4F036F61D8C54C9A821FDD52FBF96A40
2016-05-17 22:37:41,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4F036F61D8C54C9A821FDD52FBF96A40
2016-05-17 22:37:41,765 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DECCB84D380A4C5F8B3DD65DCD6BDEC8, ordinal: 0
2016-05-17 22:37:41,765 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DECCB84D380A4C5F8B3DD65DCD6BDEC8, mem on start (mb), free: 272, total: 386, max: 672
2016-05-17 22:37:41,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:41,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:41,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:41,768 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:41,768 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AC777E1E57F949C98EAFF057F29CBA66
2016-05-17 22:37:41,768 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:41,768 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:41,768 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:41,769 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:41,769 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:41,769 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:41,769 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:41,769 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:41,769 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AC777E1E57F949C98EAFF057F29CBA66, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:41,769 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4F036F61D8C54C9A821FDD52FBF96A40, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:41,769 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:41,769 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4F036F61D8C54C9A821FDD52FBF96A40 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:41,769 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:41,770 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:41,770 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4F036F61D8C54C9A821FDD52FBF96A40
2016-05-17 22:37:41,770 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:41,770 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4F036F61D8C54C9A821FDD52FBF96A40
2016-05-17 22:37:41,770 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:41,770 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524661542_0001_1_00_000000_0
2016-05-17 22:37:41,773 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:41,773 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:41,774 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 4F036F61D8C54C9A821FDD52FBF96A40
2016-05-17 22:37:41,775 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL"], id: AC777E1E57F949C98EAFF057F29CBA66
2016-05-17 22:37:41,775 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4a9452be
2016-05-17 22:37:41,775 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL"]
2016-05-17 22:37:41,775 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:41,775 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:41,783 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:41,783 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:41,783 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4a9452be
2016-05-17 22:37:41,784 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:41,785 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:41,785 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DECCB84D380A4C5F8B3DD65DCD6BDEC8, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:41,787 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:41,788 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:41,789 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:41,790 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:41,790 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:41,790 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524661542_0001_1_00_000000_0
2016-05-17 22:37:41,790 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524661542_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:41,791 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524661542_0001_1_00_000000_0
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:41,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661542_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DECCB84D380A4C5F8B3DD65DCD6BDEC8, taskAttemptId=attempt_1463524661542_0001_1_00_000000_0, startTime=1463524661736, finishTime=1463524661793, timeTaken=57, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524661786, Process_Duration=5, Process_End_Time=1463524661791, Read_Duration=1, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:37:41,794 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524661542_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:37:41,794 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524661542_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:37:41,795 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661542_0001_1][Event:TASK_FINISHED]: vertexName=DECCB84D380A4C5F8B3DD65DCD6BDEC8, taskId=task_1463524661542_0001_1_00_000000, startTime=1463524661736, finishTime=1463524661794, timeTaken=58, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
], counters=Counters: 0
2016-05-17 22:37:41,795 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524661542_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:37:41,795 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8] : 1
2016-05-17 22:37:41,795 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8] because task failed: task_1463524661542_0001_1_00_000000
2016-05-17 22:37:41,795 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:37:41,795 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:37:41,796 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8]
2016-05-17 22:37:41,796 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=AC777E1E57F949C98EAFF057F29CBA66, vertexId=vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8]
2016-05-17 22:37:41,797 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:41,798 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:41,798 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:41,799 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:41,800 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661542_0001_1][Event:VERTEX_FINISHED]: vertexName=DECCB84D380A4C5F8B3DD65DCD6BDEC8, vertexId=vertex_1463524661542_0001_1_00, initRequestedTime=1463524661710, initedTime=1463524661724, startRequestedTime=1463524661725, startedTime=1463524661725, finishTime=1463524661799, timeTaken=74, status=FAILED, diagnostics=Task failed, taskId=task_1463524661542_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524661736, firstTasksToStart=[ task_1463524661542_0001_1_00_000000 ], lastTaskFinishTime=1463524661793, lastTasksToFinish=[ task_1463524661542_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:37:41,800 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:37:41,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 19 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524661786, Process_Duration=5, Process_End_Time=1463524661791, Read_Duration=1, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:37:41,801 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DECCB84D380A4C5F8B3DD65DCD6BDEC8, mem on close (mb), free: 269, total: 386, max: 672
2016-05-17 22:37:41,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=DECCB84D380A4C5F8B3DD65DCD6BDEC8, index=0
2016-05-17 22:37:41,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=DECCB84D380A4C5F8B3DD65DCD6BDEC8, sourceVertex=4F036F61D8C54C9A821FDD52FBF96A40
2016-05-17 22:37:41,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=DECCB84D380A4C5F8B3DD65DCD6BDEC8, sourceVertex=AC777E1E57F949C98EAFF057F29CBA66
2016-05-17 22:37:41,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:41,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661542_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524661542_0001_00_000001, stoppedTime=1463524661802, exitStatus=0
2016-05-17 22:37:41,802 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524661542_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:41,802 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524661542_0001_1_00_000000_0 failed: true container: container_1463524661542_0001_00_000001 numFailedTAs: 0
2016-05-17 22:37:41,802 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:37:41,803 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:37:41,803 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524661542_0001_1
2016-05-17 22:37:41,803 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: AC777E1E57F949C98EAFF057F29CBA66 for vertex: vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8]
2016-05-17 22:37:41,804 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524661542_0001_1, queueSize=0
2016-05-17 22:37:41,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:41,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524661542_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524661542_0001_1, startTime=1463524661710, finishTime=1463524661803, timeTaken=93, status=FAILED, diagnostics=Vertex failed, vertexName=DECCB84D380A4C5F8B3DD65DCD6BDEC8, vertexId=vertex_1463524661542_0001_1_00, diagnostics=[Task failed, taskId=task_1463524661542_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524661542_0001_1_00 [DECCB84D380A4C5F8B3DD65DCD6BDEC8] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:37:41,805 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524661542_0001_1 finished with state: FAILED
2016-05-17 22:37:41,805 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524661542_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:41,805 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524661542_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:37:41 Completed Dag: dag_1463524661542_0001_1
2016-05-17 22:37:41,806 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524661542_0001_1, dagState=FAILED
2016-05-17 22:37:41,806 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524661542_0001_1_post
2016-05-17 22:37:41,806 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:41,806 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:41,806 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524661542_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:37:41,806 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524661542_0001_00_000001
2016-05-17 22:37:41,807 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524661542_0001_00_000001
2016-05-17 22:37:41,810 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524661542_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:37:41,810 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524661542_0001_00_000001 completed but with errors
2016-05-17 22:37:41,810 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524661542_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:37:41,810 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524661542_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:37:41,810 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524661542_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:37:41,824 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.-1_VALUES_ALL, applicationId=application_1463524661542_0001
2016-05-17 22:37:41,826 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:41,826 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:41,826 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:41,826 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:41,827 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:41,827 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:41,829 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:41,829 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:41,829 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:41,829 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:41,830 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:41,830 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:41,830 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:41,835 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/01CCC59F717C4FFA90761E41CF924967/C2B7D16053794330AD340F4478EEEF04/.tez/application_1463524661542_0001
2016-05-17 22:37:41,835 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-17 22:37:41,835 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...ld'_RESULTS.-1_VALUES_ALL
2016-05-17 22:37:41,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-17 22:37:41,836 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_VALUES_ALL/_temporary
2016-05-17 22:37:41,837 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.315
2016-05-17 22:37:41,837 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_VALUES_ALL
2016-05-17 22:37:41,861 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:41,862 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:41,893 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-17 22:37:41,894 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-17 22:37:41,895 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:41,895 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:41,895 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_ARGS_'field2'
2016-05-17 22:37:41,920 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:41,920 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:41,962 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 22:37:41,967 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-17 22:37:41,968 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:41,968 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:41,968 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:41,981 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:41,982 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS
2016-05-17 22:37:41,995 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:41,995 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:41,995 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS"]
2016-05-17 22:37:41,996 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:41,996 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:41,997 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:41,998 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-17 22:37:41,998 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:42,010 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:42,015 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:42,015 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:42,015 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:42,026 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/62E7138AA90141B98C96AB578ACBAD14/B6D21FBC534C4F1AA2D147C2C5E808B1/.tez/application_1463524662015_0001 doesn't exist and is created
2016-05-17 22:37:42,045 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:42,045 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:42,046 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/62E7138AA90141B98C96AB578ACBAD14/B6D21FBC534C4F1AA2D147C2C5E808B1/.tez/application_1463524662015_0001
2016-05-17 22:37:42,050 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524662015_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:42,060 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:42,061 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:42,063 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:42,063 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:42,063 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/62E7138AA90141B98C96AB578ACBAD14/B6D21FBC534C4F1AA2D147C2C5E808B1/.tez/application_1463524662015_0001/localmode-log-dir
2016-05-17 22:37:42,064 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/62E7138AA90141B98C96AB578ACBAD14/B6D21FBC534C4F1AA2D147C2C5E808B1/.tez/application_1463524662015_0001/localmode-log-dir/history.txt.appattempt_1463524662015_0001_000000, maxErrors=10
2016-05-17 22:37:42,064 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:42,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524662015_0001_000000, appSubmitTime=1463524662049, launchTime=1463524662049
2016-05-17 22:37:42,067 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:42,070 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:42,071 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:42,071 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:42,075 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55598
2016-05-17 22:37:42,077 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:42,082 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524662015_0001_000000, startTime=1463524662082
2016-05-17 22:37:42,082 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:42,082 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:42,146 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:42,146 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:42,146 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_ARGS_RESULTS, applicationId=application_1463524662015_0001, dagName=[62E7138AA90141B98C96AB578ACBAD14/B6D21FBC534C4F1AA2D147C2C5E808B1] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-17 22:37:42,164 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [62E7138AA90141B98C96AB578ACBAD14/B6D21FBC534C4F1AA2D147C2C5E808B1] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-17 22:37:42,165 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524662015_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/62E7138AA90141B98C96AB578ACBAD14/B6D21FBC534C4F1AA2D147C2C5E808B1/.tez/application_1463524662015_0001/localmode-log-dir/dag_1463524662015_0001_1.dot
2016-05-17 22:37:42,165 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/62E7138AA90141B98C96AB578ACBAD14/B6D21FBC534C4F1AA2D147C2C5E808B1/.tez/application_1463524662015_0001/localmode-log-dir/dag_1463524662015_0001_1-tez-dag.pb.txt
2016-05-17 22:37:42,171 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524662015_0001_1
2016-05-17 22:37:42,171 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:42,171 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [62E7138AA90141B98C96AB578ACBAD14/B6D21FBC534C4F1AA2D147C2C5E808B1] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-17 22:37:42 Running Dag: dag_1463524662015_0001_1
2016-05-17 22:37:42,182 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662015_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524662015_0001_1, submitTime=1463524662164
2016-05-17 22:37:42,182 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:42,183 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FB18428579D14851962D567C812919E3
2016-05-17 22:37:42,184 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:42,184 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662015_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524662015_0001_1, initTime=1463524662183
2016-05-17 22:37:42,185 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524662015_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:42,184 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:42,186 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662015_0001_1][Event:DAG_STARTED]: dagID=dag_1463524662015_0001_1, startTime=1463524662186
2016-05-17 22:37:42,186 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:42,186 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524662015_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:42,186 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FB18428579D14851962D567C812919E3 : {10757EF613EF4E24AEA9B6F28F8CCEC2={InputName=10757EF613EF4E24AEA9B6F28F8CCEC2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:42,186 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 10757EF613EF4E24AEA9B6F28F8CCEC2, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:42,187 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3]
2016-05-17 22:37:42,187 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3]
2016-05-17 22:37:42,187 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3]
2016-05-17 22:37:42,187 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3]. Starting root input initializers: 1
2016-05-17 22:37:42,188 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:42,186 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:42,189 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 10757EF613EF4E24AEA9B6F28F8CCEC2 on vertex vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3]
2016-05-17 22:37:42,190 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...'_RESULTS.-1_ARGS_RESULTS, applicationId=application_1463524662015_0001, dagName=[62E7138AA90141B98C96AB578ACBAD14/B6D21FBC534C4F1AA2D147C2C5E808B1] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS
2016-05-17 22:37:42,192 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:42,193 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 10757EF613EF4E24AEA9B6F28F8CCEC2 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:42,194 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:42,194 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:42,196 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:42,196 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 10757EF613EF4E24AEA9B6F28F8CCEC2 on vertex vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3]
2016-05-17 22:37:42,197 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:42,197 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {10757EF613EF4E24AEA9B6F28F8CCEC2=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:42,197 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3] parallelism set to 1
2016-05-17 22:37:42,197 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:42,198 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662015_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524662015_0001_1_00, eventCount=1
2016-05-17 22:37:42,198 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3]
2016-05-17 22:37:42,198 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A49BE9D0563B4A23BD340A371450F2A0, vertexId=vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:42,198 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A49BE9D0563B4A23BD340A371450F2A0, vertexId=vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3]
2016-05-17 22:37:42,200 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A49BE9D0563B4A23BD340A371450F2A0, vertexName=FB18428579D14851962D567C812919E3, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:42,200 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A49BE9D0563B4A23BD340A371450F2A0, vertexId=vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3]
2016-05-17 22:37:42,201 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:42,202 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662015_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FB18428579D14851962D567C812919E3, vertexId=vertex_1463524662015_0001_1_00, initRequestedTime=1463524662186, initedTime=1463524662201, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:42,202 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3] with distanceFromRoot: 0
2016-05-17 22:37:42,202 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:42,202 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FB18428579D14851962D567C812919E3
2016-05-17 22:37:42,202 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3]
2016-05-17 22:37:42,203 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:42,203 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662015_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524662015_0001_1_00, startRequestedTime=1463524662202, startedTime=1463524662202
2016-05-17 22:37:42,203 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:42,203 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:42,204 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662015_0001_1][Event:TASK_STARTED]: vertexName=FB18428579D14851962D567C812919E3, taskId=task_1463524662015_0001_1_00_000000, scheduledTime=1463524662203, launchTime=1463524662203
2016-05-17 22:37:42,204 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524662015_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:42,204 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3]
2016-05-17 22:37:42,204 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [62E7138AA90141B98C96AB578ACBAD14/B6D21FBC534C4F1AA2D147C2C5E808B1] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS, VertexName: FB18428579D14851962D567C812919E3, VertexParallelism: 1, TaskAttemptID:attempt_1463524662015_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=10757EF613EF4E24AEA9B6F28F8CCEC2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A49BE9D0563B4A23BD340A371450F2A0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:42,205 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524662015_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:42,205 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:42,206 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:42,206 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:42,206 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:42,207 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524662015_0001_00_000001
2016-05-17 22:37:42,207 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524662015_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:42,208 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524662015_0001_00_000001
2016-05-17 22:37:42,208 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:42,209 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524662015_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:42,209 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:42,211 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524662015_0001, with dag id: dag_1463524662015_0001_1
2016-05-17 22:37:42,211 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662015_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524662015_0001_00_000001, launchTime=1463524662209
2016-05-17 22:37:42,214 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524662015_0001_00_000001
2016-05-17 22:37:42,216 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524662015_0001_1_00_000000_0] to container: [container_1463524662015_0001_00_000001]
2016-05-17 22:37:42,216 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524662015_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:42,217 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524662015_0001_00_000001 given task: attempt_1463524662015_0001_1_00_000000_0
2016-05-17 22:37:42,217 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524662015_0001_1_00_000000_0] started. Is using containerId: [container_1463524662015_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:42,217 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662015_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FB18428579D14851962D567C812919E3, taskAttemptId=attempt_1463524662015_0001_1_00_000000_0, startTime=1463524662217, containerId=container_1463524662015_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524662015_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:42,218 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524662015_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:42,218 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524662015_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:42,217 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524662015_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524662015_0001_1_00_000000_0
2016-05-17 22:37:42,218 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:42 Starting to run new task attempt: attempt_1463524662015_0001_1_00_000000_0
2016-05-17 22:37:42,219 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524662015_0001_1_00_000000_0
2016-05-17 22:37:42,219 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:42,220 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:42,220 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:42,220 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:42,220 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:42,221 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:42,221 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [62E7138AA90141B98C96AB578ACBAD14/B6D21FBC534C4F1AA2D147C2C5E808B1] every.every/(1/1) ...'_RESULTS.-1_ARGS_RESULTS, VertexName: FB18428579D14851962D567C812919E3, VertexParallelism: 1, TaskAttemptID:attempt_1463524662015_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=10757EF613EF4E24AEA9B6F28F8CCEC2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A49BE9D0563B4A23BD340A371450F2A0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:42,221 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:42,222 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524662015_0001_1_00_000000_0
2016-05-17 22:37:42,223 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:42,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:42,224 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524662015_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:42,226 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=10757EF613EF4E24AEA9B6F28F8CCEC2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:42,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:42,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 10757EF613EF4E24AEA9B6F28F8CCEC2
2016-05-17 22:37:42,227 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 10757EF613EF4E24AEA9B6F28F8CCEC2
2016-05-17 22:37:42,229 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:42,229 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A49BE9D0563B4A23BD340A371450F2A0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:42,229 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:42,230 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A49BE9D0563B4A23BD340A371450F2A0
2016-05-17 22:37:42,230 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:42,230 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A49BE9D0563B4A23BD340A371450F2A0
2016-05-17 22:37:42,231 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:42,239 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:42,239 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 10757EF613EF4E24AEA9B6F28F8CCEC2
2016-05-17 22:37:42,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 10757EF613EF4E24AEA9B6F28F8CCEC2
2016-05-17 22:37:42,243 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FB18428579D14851962D567C812919E3, ordinal: 0
2016-05-17 22:37:42,244 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FB18428579D14851962D567C812919E3, mem on start (mb), free: 194, total: 386, max: 672
2016-05-17 22:37:42,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:42,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:42,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:42,245 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:42,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A49BE9D0563B4A23BD340A371450F2A0
2016-05-17 22:37:42,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:42,246 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:42,246 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:42,246 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:42,247 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:42,247 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:42,247 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:42,248 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:42,248 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 10757EF613EF4E24AEA9B6F28F8CCEC2, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:42,248 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A49BE9D0563B4A23BD340A371450F2A0, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:42,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:42,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 10757EF613EF4E24AEA9B6F28F8CCEC2
2016-05-17 22:37:42,249 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:42,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 10757EF613EF4E24AEA9B6F28F8CCEC2
2016-05-17 22:37:42,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 10757EF613EF4E24AEA9B6F28F8CCEC2 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:42,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:42,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:42,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:42,251 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524662015_0001_1_00_000000_0
2016-05-17 22:37:42,254 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:42,255 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 10757EF613EF4E24AEA9B6F28F8CCEC2
2016-05-17 22:37:42,255 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS"], id: A49BE9D0563B4A23BD340A371450F2A0
2016-05-17 22:37:42,255 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS"]
2016-05-17 22:37:42,255 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:42,256 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:42,256 INFO input.MRInput (MRInput.java:checkAndAwaitRecordReaderInitialization(597)) - Awaiting RecordReader initialization
2016-05-17 22:37:42,256 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:42,256 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@e1bf3ff
2016-05-17 22:37:42,257 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@e1bf3ff
2016-05-17 22:37:42,258 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:42,258 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:42,258 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FB18428579D14851962D567C812919E3, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:42,261 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:42,262 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:42,262 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524662015_0001_1_00_000000_0
2016-05-17 22:37:42,262 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524662015_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:42,263 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524662015_0001_1_00_000000_0
2016-05-17 22:37:42,264 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FB18428579D14851962D567C812919E3, mem on close (mb), free: 192, total: 386, max: 672
2016-05-17 22:37:42 Completed running task attempt: attempt_1463524662015_0001_1_00_000000_0
2016-05-17 22:37:42,265 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524662015_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:42,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524662258, Process_Duration=5, Process_End_Time=1463524662263, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:42,267 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662015_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FB18428579D14851962D567C812919E3, taskAttemptId=attempt_1463524662015_0001_1_00_000000_0, startTime=1463524662217, finishTime=1463524662266, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524662258, Process_Duration=5, Process_End_Time=1463524662263, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:42,267 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:42,267 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:42,267 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:42,267 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524662015_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:42,267 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524662015_0001_00_000001
2016-05-17 22:37:42,268 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524662015_0001_1_00_000000_0
2016-05-17 22:37:42,268 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:42,268 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:42,269 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662015_0001_1][Event:TASK_FINISHED]: vertexName=FB18428579D14851962D567C812919E3, taskId=task_1463524662015_0001_1_00_000000, startTime=1463524662217, finishTime=1463524662268, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1463524662015_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524662258, Process_Duration=5, Process_End_Time=1463524662263, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:42,269 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524662015_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:42,269 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3] : 1
2016-05-17 22:37:42,269 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3]
2016-05-17 22:37:42,270 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:42,270 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662015_0001_1][Event:VERTEX_FINISHED]: vertexName=FB18428579D14851962D567C812919E3, vertexId=vertex_1463524662015_0001_1_00, initRequestedTime=1463524662186, initedTime=1463524662201, startRequestedTime=1463524662202, startedTime=1463524662202, finishTime=1463524662269, timeTaken=67, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524662258, Process_Duration=5, Process_End_Time=1463524662263, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524662217, firstTasksToStart=[ task_1463524662015_0001_1_00_000000 ], lastTaskFinishTime=1463524662266, lastTasksToFinish=[ task_1463524662015_0001_1_00_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524662015_0001_1_00_000000 ], longestDurationTasks=[ task_1463524662015_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:42,270 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:42,270 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524662015_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:42,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662015_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524662015_0001_00_000001, stoppedTime=1463524662271, exitStatus=0
2016-05-17 22:37:42,271 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524662015_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:42,271 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:42,271 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:42,271 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524662015_0001_1
2016-05-17 22:37:42,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662015_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524662015_0001_1
2016-05-17 22:37:42,272 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A49BE9D0563B4A23BD340A371450F2A0 for vertex: vertex_1463524662015_0001_1_00 [FB18428579D14851962D567C812919E3]
2016-05-17 22:37:42,278 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524662015_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:42,278 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524662015_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:42,278 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524662015_0001_00_000001, Exiting
2016-05-17 22:37:42,279 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524662015_0001_00_000001
2016-05-17 22:37:42,279 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524662015_0001_00_000001
2016-05-17 22:37:42,281 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524662015_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:42,281 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524662015_0001_00_000001 completed successfully
2016-05-17 22:37:42,281 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524662015_0001_1, queueSize=0
2016-05-17 22:37:42,282 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:42,282 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662015_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524662015_0001_1, startTime=1463524662186, finishTime=1463524662280, timeTaken=94, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524662258, Process_Duration=5, Process_End_Time=1463524662263, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:42,282 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524662015_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:42,282 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524662015_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:42,283 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524662015_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:42,283 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524662015_0001_00_000001
2016-05-17 22:37:42 Completed Dag: dag_1463524662015_0001_1
2016-05-17 22:37:42,283 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524662015_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:42,283 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524662015_0001_1_post
2016-05-17 22:37:42,283 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:42,284 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:42,289 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...'_RESULTS.-1_ARGS_RESULTS, applicationId=application_1463524662015_0001
2016-05-17 22:37:42,291 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:42,291 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:42,291 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:42,292 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:42,292 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:42,292 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:42,293 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:42,293 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:42,293 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:42,294 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:42,294 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:42,294 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:42,295 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:42,299 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/62E7138AA90141B98C96AB578ACBAD14/B6D21FBC534C4F1AA2D147C2C5E808B1/.tez/application_1463524662015_0001
2016-05-17 22:37:42,299 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.-1_ARGS_RESULTS/_temporary
2016-05-17 22:37:42,300 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.304
2016-05-17 22:37:42,371 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_ARGS_RESULTS
2016-05-17 22:37:42,395 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:42,395 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:42,396 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:42,397 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:42,426 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 22:37:42,433 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.010
2016-05-17 22:37:42,433 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:42,434 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:42,434 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.-1_ARGS_ALL
2016-05-17 22:37:42,469 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:42,469 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:42,524 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 22:37:42,551 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.040
2016-05-17 22:37:42,552 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:42,552 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:42,552 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:42,576 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:42,577 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1
2016-05-17 22:37:42,592 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:42,593 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:42,593 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1"]
2016-05-17 22:37:42,593 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:42,593 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:42,593 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:42,594 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-17 22:37:42,594 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:42,612 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:42,620 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:42,621 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:42,621 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:42,633 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2336356673634A0FA60460AD5F912C77/CBFB55B47C6A47F4A33A93AC7724B4C6/.tez/application_1463524662620_0001 doesn't exist and is created
2016-05-17 22:37:42,655 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:42,655 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:42,656 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2336356673634A0FA60460AD5F912C77/CBFB55B47C6A47F4A33A93AC7724B4C6/.tez/application_1463524662620_0001
2016-05-17 22:37:42,661 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524662620_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:42,674 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:42,675 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:42,677 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:42,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:42,677 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2336356673634A0FA60460AD5F912C77/CBFB55B47C6A47F4A33A93AC7724B4C6/.tez/application_1463524662620_0001/localmode-log-dir
2016-05-17 22:37:42,678 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2336356673634A0FA60460AD5F912C77/CBFB55B47C6A47F4A33A93AC7724B4C6/.tez/application_1463524662620_0001/localmode-log-dir/history.txt.appattempt_1463524662620_0001_000000, maxErrors=10
2016-05-17 22:37:42,678 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:42,678 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524662620_0001_000000, appSubmitTime=1463524662661, launchTime=1463524662661
2016-05-17 22:37:42,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:42,684 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:42,685 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:42,686 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:42,687 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49086
2016-05-17 22:37:42,692 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:42,694 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524662620_0001_000000, startTime=1463524662694
2016-05-17 22:37:42,694 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:42,694 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:42,756 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:42,756 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:42,756 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_'field2'_-1, applicationId=application_1463524662620_0001, dagName=[2336356673634A0FA60460AD5F912C77/CBFB55B47C6A47F4A33A93AC7724B4C6] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-17 22:37:42,772 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2336356673634A0FA60460AD5F912C77/CBFB55B47C6A47F4A33A93AC7724B4C6] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-17 22:37:42,774 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524662620_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2336356673634A0FA60460AD5F912C77/CBFB55B47C6A47F4A33A93AC7724B4C6/.tez/application_1463524662620_0001/localmode-log-dir/dag_1463524662620_0001_1.dot
2016-05-17 22:37:42,774 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2336356673634A0FA60460AD5F912C77/CBFB55B47C6A47F4A33A93AC7724B4C6/.tez/application_1463524662620_0001/localmode-log-dir/dag_1463524662620_0001_1-tez-dag.pb.txt
2016-05-17 22:37:42,780 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524662620_0001_1
2016-05-17 22:37:42,780 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:42,780 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2336356673634A0FA60460AD5F912C77/CBFB55B47C6A47F4A33A93AC7724B4C6] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-17 22:37:42 Running Dag: dag_1463524662620_0001_1
2016-05-17 22:37:42,792 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:42,792 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662620_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524662620_0001_1, submitTime=1463524662773
2016-05-17 22:37:42,793 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex ADA0784236344F1FBFC62595770DDE20
2016-05-17 22:37:42,793 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:42,793 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:42,793 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662620_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524662620_0001_1, initTime=1463524662792
2016-05-17 22:37:42,794 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524662620_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:42,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662620_0001_1][Event:DAG_STARTED]: dagID=dag_1463524662620_0001_1, startTime=1463524662794
2016-05-17 22:37:42,795 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:42,794 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:42,795 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524662620_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:42,795 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_'field2'_-1, applicationId=application_1463524662620_0001, dagName=[2336356673634A0FA60460AD5F912C77/CBFB55B47C6A47F4A33A93AC7724B4C6] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1
2016-05-17 22:37:42,795 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: ADA0784236344F1FBFC62595770DDE20 : {318F2C17213D4FB89C75F7C7FE0AF267={InputName=318F2C17213D4FB89C75F7C7FE0AF267}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:42,795 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 318F2C17213D4FB89C75F7C7FE0AF267, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:42,796 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20]
2016-05-17 22:37:42,796 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20]
2016-05-17 22:37:42,796 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20]
2016-05-17 22:37:42,796 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20]. Starting root input initializers: 1
2016-05-17 22:37:42,797 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:42,797 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 318F2C17213D4FB89C75F7C7FE0AF267 on vertex vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20]
2016-05-17 22:37:42,799 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:42,799 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 318F2C17213D4FB89C75F7C7FE0AF267 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:42,799 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:42,799 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:42,801 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:42,801 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 318F2C17213D4FB89C75F7C7FE0AF267 on vertex vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20]
2016-05-17 22:37:42,801 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:42,801 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {318F2C17213D4FB89C75F7C7FE0AF267=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:42,802 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20] parallelism set to 1
2016-05-17 22:37:42,802 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:42,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662620_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524662620_0001_1_00, eventCount=1
2016-05-17 22:37:42,802 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20]
2016-05-17 22:37:42,802 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6574B2F72DAC4FD9AF0716FC7AD332CD, vertexId=vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:42,803 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6574B2F72DAC4FD9AF0716FC7AD332CD, vertexId=vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20]
2016-05-17 22:37:42,804 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6574B2F72DAC4FD9AF0716FC7AD332CD, vertexName=ADA0784236344F1FBFC62595770DDE20, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:42,804 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6574B2F72DAC4FD9AF0716FC7AD332CD, vertexId=vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20]
2016-05-17 22:37:42,805 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:42,806 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662620_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ADA0784236344F1FBFC62595770DDE20, vertexId=vertex_1463524662620_0001_1_00, initRequestedTime=1463524662795, initedTime=1463524662805, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:42,806 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20] with distanceFromRoot: 0
2016-05-17 22:37:42,806 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:42,806 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in ADA0784236344F1FBFC62595770DDE20
2016-05-17 22:37:42,806 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20]
2016-05-17 22:37:42,807 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:42,807 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662620_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524662620_0001_1_00, startRequestedTime=1463524662806, startedTime=1463524662806
2016-05-17 22:37:42,807 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:42,808 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:42,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662620_0001_1][Event:TASK_STARTED]: vertexName=ADA0784236344F1FBFC62595770DDE20, taskId=task_1463524662620_0001_1_00_000000, scheduledTime=1463524662808, launchTime=1463524662808
2016-05-17 22:37:42,808 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524662620_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:42,808 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20]
2016-05-17 22:37:42,809 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2336356673634A0FA60460AD5F912C77/CBFB55B47C6A47F4A33A93AC7724B4C6] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1, VertexName: ADA0784236344F1FBFC62595770DDE20, VertexParallelism: 1, TaskAttemptID:attempt_1463524662620_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=318F2C17213D4FB89C75F7C7FE0AF267, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=6574B2F72DAC4FD9AF0716FC7AD332CD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:42,809 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524662620_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:42,809 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:42,811 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:42,811 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:42,811 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:42,812 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524662620_0001_00_000001
2016-05-17 22:37:42,812 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524662620_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:42,813 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524662620_0001_00_000001
2016-05-17 22:37:42,813 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:42,814 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524662620_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:42,814 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:42,814 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662620_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524662620_0001_00_000001, launchTime=1463524662814
2016-05-17 22:37:42,815 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524662620_0001_00_000001
2016-05-17 22:37:42,816 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524662620_0001_1_00_000000_0] to container: [container_1463524662620_0001_00_000001]
2016-05-17 22:37:42,816 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524662620_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:42,816 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524662620_0001_1_00_000000_0] started. Is using containerId: [container_1463524662620_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:42,817 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:42,817 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662620_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ADA0784236344F1FBFC62595770DDE20, taskAttemptId=attempt_1463524662620_0001_1_00_000000_0, startTime=1463524662816, containerId=container_1463524662620_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524662620_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:42,817 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524662620_0001, with dag id: dag_1463524662620_0001_1
2016-05-17 22:37:42,817 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524662620_0001_00_000001 given task: attempt_1463524662620_0001_1_00_000000_0
2016-05-17 22:37:42,817 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524662620_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:42,818 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524662620_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524662620_0001_1_00_000000_0
2016-05-17 22:37:42 Starting to run new task attempt: attempt_1463524662620_0001_1_00_000000_0
2016-05-17 22:37:42,818 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524662620_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:42,818 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524662620_0001_1_00_000000_0
2016-05-17 22:37:42,819 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:42,819 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:42,820 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:42,820 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:42,820 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:42,820 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:42,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2336356673634A0FA60460AD5F912C77/CBFB55B47C6A47F4A33A93AC7724B4C6] every.every/(1/1) ...ld'_RESULTS.0_'field2'_-1, VertexName: ADA0784236344F1FBFC62595770DDE20, VertexParallelism: 1, TaskAttemptID:attempt_1463524662620_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=318F2C17213D4FB89C75F7C7FE0AF267, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=6574B2F72DAC4FD9AF0716FC7AD332CD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:42,821 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:42,821 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524662620_0001_1_00_000000_0
2016-05-17 22:37:42,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:42,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:42,824 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524662620_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:42,825 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:42,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=318F2C17213D4FB89C75F7C7FE0AF267, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:42,825 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:42,825 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 318F2C17213D4FB89C75F7C7FE0AF267
2016-05-17 22:37:42,826 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 318F2C17213D4FB89C75F7C7FE0AF267
2016-05-17 22:37:42,826 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6574B2F72DAC4FD9AF0716FC7AD332CD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:42,826 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:42,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6574B2F72DAC4FD9AF0716FC7AD332CD
2016-05-17 22:37:42,827 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:42,827 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6574B2F72DAC4FD9AF0716FC7AD332CD
2016-05-17 22:37:42,831 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:42,831 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 318F2C17213D4FB89C75F7C7FE0AF267
2016-05-17 22:37:42,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 318F2C17213D4FB89C75F7C7FE0AF267
2016-05-17 22:37:42,837 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:42,844 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ADA0784236344F1FBFC62595770DDE20, ordinal: 0
2016-05-17 22:37:42,844 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADA0784236344F1FBFC62595770DDE20, mem on start (mb), free: 309, total: 385, max: 672
2016-05-17 22:37:42,844 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:42,844 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:42,845 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:42,846 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:42,846 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6574B2F72DAC4FD9AF0716FC7AD332CD
2016-05-17 22:37:42,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:42,847 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:42,847 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:42,847 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:42,848 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:42,848 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:42,848 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:42,849 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:42,849 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 318F2C17213D4FB89C75F7C7FE0AF267, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:42,849 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6574B2F72DAC4FD9AF0716FC7AD332CD, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:42,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:42,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 318F2C17213D4FB89C75F7C7FE0AF267 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:42,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 318F2C17213D4FB89C75F7C7FE0AF267
2016-05-17 22:37:42,850 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:42,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:42,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:42,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 318F2C17213D4FB89C75F7C7FE0AF267
2016-05-17 22:37:42,852 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:42,852 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524662620_0001_1_00_000000_0
2016-05-17 22:37:42,852 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:42,853 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@221e8d6a
2016-05-17 22:37:42,854 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@221e8d6a
2016-05-17 22:37:42,854 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:42,855 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:42,856 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:42,856 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 318F2C17213D4FB89C75F7C7FE0AF267
2016-05-17 22:37:42,857 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1"], id: 6574B2F72DAC4FD9AF0716FC7AD332CD
2016-05-17 22:37:42,857 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1"]
2016-05-17 22:37:42,857 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:42,857 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:42,858 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ADA0784236344F1FBFC62595770DDE20, all 1 inputs ready in: 00:00:00.001
2016-05-17 22:37:42,861 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:42,861 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:42,862 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524662620_0001_1_00_000000_0
2016-05-17 22:37:42,862 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524662620_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:42,863 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524662620_0001_1_00_000000_0
2016-05-17 22:37:42,863 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADA0784236344F1FBFC62595770DDE20, mem on close (mb), free: 308, total: 385, max: 672
2016-05-17 22:37:42 Completed running task attempt: attempt_1463524662620_0001_1_00_000000_0
2016-05-17 22:37:42,864 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524662620_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:42,865 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662620_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ADA0784236344F1FBFC62595770DDE20, taskAttemptId=attempt_1463524662620_0001_1_00_000000_0, startTime=1463524662816, finishTime=1463524662865, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524662858, Process_Duration=5, Process_End_Time=1463524662863, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:42,866 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524662620_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:42,866 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:42,866 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524662620_0001_1_00_000000_0
2016-05-17 22:37:42,866 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:42,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662620_0001_1][Event:TASK_FINISHED]: vertexName=ADA0784236344F1FBFC62595770DDE20, taskId=task_1463524662620_0001_1_00_000000, startTime=1463524662816, finishTime=1463524662866, timeTaken=50, status=SUCCEEDED, successfulAttemptID=attempt_1463524662620_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524662858, Process_Duration=5, Process_End_Time=1463524662863, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:42,867 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524662620_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:42,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:42,867 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20] : 1
2016-05-17 22:37:42,867 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20]
2016-05-17 22:37:42,868 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662620_0001_1][Event:VERTEX_FINISHED]: vertexName=ADA0784236344F1FBFC62595770DDE20, vertexId=vertex_1463524662620_0001_1_00, initRequestedTime=1463524662795, initedTime=1463524662805, startRequestedTime=1463524662806, startedTime=1463524662806, finishTime=1463524662867, timeTaken=61, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524662858, Process_Duration=5, Process_End_Time=1463524662863, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524662816, firstTasksToStart=[ task_1463524662620_0001_1_00_000000 ], lastTaskFinishTime=1463524662865, lastTasksToFinish=[ task_1463524662620_0001_1_00_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524662620_0001_1_00_000000 ], longestDurationTasks=[ task_1463524662620_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:42,869 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:42,869 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524662620_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:42,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524662858, Process_Duration=5, Process_End_Time=1463524662863, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:42,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662620_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524662620_0001_00_000001, stoppedTime=1463524662869, exitStatus=0
2016-05-17 22:37:42,869 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:42,870 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:42,870 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:42,869 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524662620_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:42,870 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:42,870 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524662620_0001_1
2016-05-17 22:37:42,870 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:42,871 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662620_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524662620_0001_1
2016-05-17 22:37:42,871 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 6574B2F72DAC4FD9AF0716FC7AD332CD for vertex: vertex_1463524662620_0001_1_00 [ADA0784236344F1FBFC62595770DDE20]
2016-05-17 22:37:42,871 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524662620_0001_00_000001
2016-05-17 22:37:42,872 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524662620_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:42,872 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524662620_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:42,872 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524662620_0001_00_000001, Exiting
2016-05-17 22:37:42,872 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524662620_0001_00_000001
2016-05-17 22:37:42,873 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524662620_0001_00_000001
2016-05-17 22:37:42,874 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524662620_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:42,874 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524662620_0001_00_000001 completed successfully
2016-05-17 22:37:42,882 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524662620_0001_1, queueSize=0
2016-05-17 22:37:42,882 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:42,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524662620_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524662620_0001_1, startTime=1463524662794, finishTime=1463524662881, timeTaken=87, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524662858, Process_Duration=5, Process_End_Time=1463524662863, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:42,883 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524662620_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:42,883 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524662620_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:42,883 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524662620_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:42,883 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524662620_0001_00_000001
2016-05-17 22:37:42 Completed Dag: dag_1463524662620_0001_1
2016-05-17 22:37:42,884 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524662620_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:42,884 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524662620_0001_1_post
2016-05-17 22:37:42,885 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:42,885 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:42,890 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.0_'field2'_-1, applicationId=application_1463524662620_0001
2016-05-17 22:37:42,892 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:42,892 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:42,892 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:42,892 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:42,892 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:42,892 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:42,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:42,893 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:42,893 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:42,893 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:42,893 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:42,894 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:42,894 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:42,897 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/2336356673634A0FA60460AD5F912C77/CBFB55B47C6A47F4A33A93AC7724B4C6/.tez/application_1463524662620_0001
2016-05-17 22:37:42,898 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_-1/_temporary
2016-05-17 22:37:42,898 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.306
2016-05-17 22:37:42,953 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:42,953 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:42,974 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_-1
2016-05-17 22:37:42,994 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:42,994 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:43,025 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 22:37:43,035 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 22:37:43,035 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:43,036 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:43,036 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:43,050 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:43,050 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'
2016-05-17 22:37:43,061 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:43,061 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:43,061 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'"]
2016-05-17 22:37:43,061 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:43,061 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:43,061 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:43,062 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...SULTS.0_'field2'_'field2'
2016-05-17 22:37:43,062 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:43,074 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:43,078 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:43,078 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:43,078 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:43,087 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B0B7FE22B0AE463A9022BFCEA6785A70/F439AE461D0645DEA7503537DD310C50/.tez/application_1463524663078_0001 doesn't exist and is created
2016-05-17 22:37:43,103 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:43,104 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:43,104 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B0B7FE22B0AE463A9022BFCEA6785A70/F439AE461D0645DEA7503537DD310C50/.tez/application_1463524663078_0001
2016-05-17 22:37:43,107 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524663078_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:43,118 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:43,119 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:43,120 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:43,120 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:43,121 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B0B7FE22B0AE463A9022BFCEA6785A70/F439AE461D0645DEA7503537DD310C50/.tez/application_1463524663078_0001/localmode-log-dir
2016-05-17 22:37:43,121 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B0B7FE22B0AE463A9022BFCEA6785A70/F439AE461D0645DEA7503537DD310C50/.tez/application_1463524663078_0001/localmode-log-dir/history.txt.appattempt_1463524663078_0001_000000, maxErrors=10
2016-05-17 22:37:43,121 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:43,121 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524663078_0001_000000, appSubmitTime=1463524663107, launchTime=1463524663107
2016-05-17 22:37:43,123 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:43,125 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:43,125 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:43,128 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58395
2016-05-17 22:37:43,128 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:43,130 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:43,133 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524663078_0001_000000, startTime=1463524663133
2016-05-17 22:37:43,133 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:43,133 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:43,204 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:43,205 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:43,205 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...SULTS.0_'field2'_'field2', applicationId=application_1463524663078_0001, dagName=[B0B7FE22B0AE463A9022BFCEA6785A70/F439AE461D0645DEA7503537DD310C50] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-17 22:37:43,219 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [B0B7FE22B0AE463A9022BFCEA6785A70/F439AE461D0645DEA7503537DD310C50] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-17 22:37:43,221 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524663078_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B0B7FE22B0AE463A9022BFCEA6785A70/F439AE461D0645DEA7503537DD310C50/.tez/application_1463524663078_0001/localmode-log-dir/dag_1463524663078_0001_1.dot
2016-05-17 22:37:43,221 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B0B7FE22B0AE463A9022BFCEA6785A70/F439AE461D0645DEA7503537DD310C50/.tez/application_1463524663078_0001/localmode-log-dir/dag_1463524663078_0001_1-tez-dag.pb.txt
2016-05-17 22:37:43,226 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524663078_0001_1
2016-05-17 22:37:43,227 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:43,227 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [B0B7FE22B0AE463A9022BFCEA6785A70/F439AE461D0645DEA7503537DD310C50] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-17 22:37:43 Running Dag: dag_1463524663078_0001_1
2016-05-17 22:37:43,238 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:43,238 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663078_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524663078_0001_1, submitTime=1463524663219
2016-05-17 22:37:43,238 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C78220693E6C4CEBBC07AE5944F333E2
2016-05-17 22:37:43,239 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:43,239 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:43,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663078_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524663078_0001_1, initTime=1463524663238
2016-05-17 22:37:43,239 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524663078_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:43,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663078_0001_1][Event:DAG_STARTED]: dagID=dag_1463524663078_0001_1, startTime=1463524663239
2016-05-17 22:37:43,240 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:43,240 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:43,240 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524663078_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:43,240 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C78220693E6C4CEBBC07AE5944F333E2 : {9B5A170D2FC546B8B2FBABE1573AC7AE={InputName=9B5A170D2FC546B8B2FBABE1573AC7AE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:43,240 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9B5A170D2FC546B8B2FBABE1573AC7AE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:43,240 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2]
2016-05-17 22:37:43,240 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2]
2016-05-17 22:37:43,240 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2]
2016-05-17 22:37:43,241 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2]. Starting root input initializers: 1
2016-05-17 22:37:43,241 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:43,241 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9B5A170D2FC546B8B2FBABE1573AC7AE on vertex vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2]
2016-05-17 22:37:43,243 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...SULTS.0_'field2'_'field2', applicationId=application_1463524663078_0001, dagName=[B0B7FE22B0AE463A9022BFCEA6785A70/F439AE461D0645DEA7503537DD310C50] every.every/(1/1) ...SULTS.0_'field2'_'field2'
2016-05-17 22:37:43,244 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:43,245 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9B5A170D2FC546B8B2FBABE1573AC7AE asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:43,246 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:43,246 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:43,248 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:43,248 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9B5A170D2FC546B8B2FBABE1573AC7AE on vertex vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2]
2016-05-17 22:37:43,249 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:43,249 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9B5A170D2FC546B8B2FBABE1573AC7AE=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:43,249 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2] parallelism set to 1
2016-05-17 22:37:43,249 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:43,250 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663078_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524663078_0001_1_00, eventCount=1
2016-05-17 22:37:43,250 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2]
2016-05-17 22:37:43,250 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2F5681C8B9664FB39D204D8A7CDEAA73, vertexId=vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:43,250 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2F5681C8B9664FB39D204D8A7CDEAA73, vertexId=vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2]
2016-05-17 22:37:43,253 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2F5681C8B9664FB39D204D8A7CDEAA73, vertexName=C78220693E6C4CEBBC07AE5944F333E2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:43,253 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2F5681C8B9664FB39D204D8A7CDEAA73, vertexId=vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2]
2016-05-17 22:37:43,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:43,255 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663078_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C78220693E6C4CEBBC07AE5944F333E2, vertexId=vertex_1463524663078_0001_1_00, initRequestedTime=1463524663240, initedTime=1463524663254, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:43,255 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2] with distanceFromRoot: 0
2016-05-17 22:37:43,255 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:43,256 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C78220693E6C4CEBBC07AE5944F333E2
2016-05-17 22:37:43,256 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2]
2016-05-17 22:37:43,257 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:43,257 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663078_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524663078_0001_1_00, startRequestedTime=1463524663256, startedTime=1463524663256
2016-05-17 22:37:43,258 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:43,259 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:43,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663078_0001_1][Event:TASK_STARTED]: vertexName=C78220693E6C4CEBBC07AE5944F333E2, taskId=task_1463524663078_0001_1_00_000000, scheduledTime=1463524663258, launchTime=1463524663258
2016-05-17 22:37:43,259 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524663078_0001, with dag id: dag_1463524663078_0001_1
2016-05-17 22:37:43,259 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524663078_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:43,260 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2]
2016-05-17 22:37:43,261 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B0B7FE22B0AE463A9022BFCEA6785A70/F439AE461D0645DEA7503537DD310C50] every.every/(1/1) ...SULTS.0_'field2'_'field2', VertexName: C78220693E6C4CEBBC07AE5944F333E2, VertexParallelism: 1, TaskAttemptID:attempt_1463524663078_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9B5A170D2FC546B8B2FBABE1573AC7AE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=2F5681C8B9664FB39D204D8A7CDEAA73, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:43,261 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524663078_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:43,261 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:43,262 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:43,264 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:43,264 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:43,264 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524663078_0001_00_000001
2016-05-17 22:37:43,265 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663078_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:43,266 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524663078_0001_00_000001
2016-05-17 22:37:43,266 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:43,267 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663078_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:43,268 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:43,268 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663078_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524663078_0001_00_000001, launchTime=1463524663267
2016-05-17 22:37:43,269 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524663078_0001_00_000001
2016-05-17 22:37:43,269 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524663078_0001_1_00_000000_0] to container: [container_1463524663078_0001_00_000001]
2016-05-17 22:37:43,269 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663078_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:43,270 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524663078_0001_00_000001 given task: attempt_1463524663078_0001_1_00_000000_0
2016-05-17 22:37:43,270 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524663078_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524663078_0001_1_00_000000_0
2016-05-17 22:37:43,270 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524663078_0001_1_00_000000_0] started. Is using containerId: [container_1463524663078_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:43,270 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663078_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C78220693E6C4CEBBC07AE5944F333E2, taskAttemptId=attempt_1463524663078_0001_1_00_000000_0, startTime=1463524663270, containerId=container_1463524663078_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524663078_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:43 Starting to run new task attempt: attempt_1463524663078_0001_1_00_000000_0
2016-05-17 22:37:43,270 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524663078_0001_1_00_000000_0
2016-05-17 22:37:43,270 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524663078_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:43,270 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:43,270 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524663078_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:43,270 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:43,271 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:43,271 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:43,271 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:43,271 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:43,271 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:43,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B0B7FE22B0AE463A9022BFCEA6785A70/F439AE461D0645DEA7503537DD310C50] every.every/(1/1) ...SULTS.0_'field2'_'field2', VertexName: C78220693E6C4CEBBC07AE5944F333E2, VertexParallelism: 1, TaskAttemptID:attempt_1463524663078_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9B5A170D2FC546B8B2FBABE1573AC7AE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=2F5681C8B9664FB39D204D8A7CDEAA73, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:43,271 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:43,272 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524663078_0001_1_00_000000_0
2016-05-17 22:37:43,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:43,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:43,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9B5A170D2FC546B8B2FBABE1573AC7AE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:43,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:43,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9B5A170D2FC546B8B2FBABE1573AC7AE
2016-05-17 22:37:43,275 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9B5A170D2FC546B8B2FBABE1573AC7AE
2016-05-17 22:37:43,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2F5681C8B9664FB39D204D8A7CDEAA73, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:43,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:43,275 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524663078_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:43,278 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:43,278 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9B5A170D2FC546B8B2FBABE1573AC7AE
2016-05-17 22:37:43,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9B5A170D2FC546B8B2FBABE1573AC7AE
2016-05-17 22:37:43,279 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:43,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:43,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2F5681C8B9664FB39D204D8A7CDEAA73
2016-05-17 22:37:43,282 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:43,282 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2F5681C8B9664FB39D204D8A7CDEAA73
2016-05-17 22:37:43,291 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C78220693E6C4CEBBC07AE5944F333E2, ordinal: 0
2016-05-17 22:37:43,292 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C78220693E6C4CEBBC07AE5944F333E2, mem on start (mb), free: 232, total: 385, max: 672
2016-05-17 22:37:43,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:43,293 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:43,293 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:43,292 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:43,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2F5681C8B9664FB39D204D8A7CDEAA73
2016-05-17 22:37:43,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:43,295 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:43,296 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:43,296 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:43,296 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:43,296 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:43,297 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:43,297 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:43,297 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9B5A170D2FC546B8B2FBABE1573AC7AE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:43,297 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2F5681C8B9664FB39D204D8A7CDEAA73, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:43,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:43,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9B5A170D2FC546B8B2FBABE1573AC7AE being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:43,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9B5A170D2FC546B8B2FBABE1573AC7AE
2016-05-17 22:37:43,298 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:43,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9B5A170D2FC546B8B2FBABE1573AC7AE
2016-05-17 22:37:43,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:43,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:43,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:43,299 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524663078_0001_1_00_000000_0
2016-05-17 22:37:43,300 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:43,301 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@32d4ddfd
2016-05-17 22:37:43,302 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@32d4ddfd
2016-05-17 22:37:43,302 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:43,302 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:43,303 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:43,303 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 9B5A170D2FC546B8B2FBABE1573AC7AE
2016-05-17 22:37:43,304 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'"], id: 2F5681C8B9664FB39D204D8A7CDEAA73
2016-05-17 22:37:43,304 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'"]
2016-05-17 22:37:43,304 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:43,304 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:43,305 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C78220693E6C4CEBBC07AE5944F333E2, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:43,308 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:43,308 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:43,308 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524663078_0001_1_00_000000_0
2016-05-17 22:37:43,309 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524663078_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:43,309 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524663078_0001_1_00_000000_0
2016-05-17 22:37:43,309 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C78220693E6C4CEBBC07AE5944F333E2, mem on close (mb), free: 231, total: 385, max: 672
2016-05-17 22:37:43 Completed running task attempt: attempt_1463524663078_0001_1_00_000000_0
2016-05-17 22:37:43,310 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524663078_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:43,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524663305, Process_Duration=4, Process_End_Time=1463524663309, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:43,312 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:43,312 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:43,311 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663078_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C78220693E6C4CEBBC07AE5944F333E2, taskAttemptId=attempt_1463524663078_0001_1_00_000000_0, startTime=1463524663270, finishTime=1463524663311, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524663305, Process_Duration=4, Process_End_Time=1463524663309, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:43,311 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:43,312 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524663078_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:43,312 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524663078_0001_00_000001
2016-05-17 22:37:43,313 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524663078_0001_1_00_000000_0
2016-05-17 22:37:43,313 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:43,313 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:43,314 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663078_0001_1][Event:TASK_FINISHED]: vertexName=C78220693E6C4CEBBC07AE5944F333E2, taskId=task_1463524663078_0001_1_00_000000, startTime=1463524663270, finishTime=1463524663313, timeTaken=43, status=SUCCEEDED, successfulAttemptID=attempt_1463524663078_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524663305, Process_Duration=4, Process_End_Time=1463524663309, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:43,314 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524663078_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:43,314 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2] : 1
2016-05-17 22:37:43,314 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2]
2016-05-17 22:37:43,315 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:43,315 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663078_0001_1][Event:VERTEX_FINISHED]: vertexName=C78220693E6C4CEBBC07AE5944F333E2, vertexId=vertex_1463524663078_0001_1_00, initRequestedTime=1463524663240, initedTime=1463524663254, startRequestedTime=1463524663256, startedTime=1463524663256, finishTime=1463524663314, timeTaken=58, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524663305, Process_Duration=4, Process_End_Time=1463524663309, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524663270, firstTasksToStart=[ task_1463524663078_0001_1_00_000000 ], lastTaskFinishTime=1463524663311, lastTasksToFinish=[ task_1463524663078_0001_1_00_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524663078_0001_1_00_000000 ], longestDurationTasks=[ task_1463524663078_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:43,315 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:43,315 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663078_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:43,315 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663078_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524663078_0001_00_000001, stoppedTime=1463524663315, exitStatus=0
2016-05-17 22:37:43,316 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:43,316 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663078_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:43,316 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:43,316 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524663078_0001_1
2016-05-17 22:37:43,317 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663078_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524663078_0001_1
2016-05-17 22:37:43,317 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2F5681C8B9664FB39D204D8A7CDEAA73 for vertex: vertex_1463524663078_0001_1_00 [C78220693E6C4CEBBC07AE5944F333E2]
2016-05-17 22:37:43,323 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524663078_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:43,323 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524663078_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:43,323 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524663078_0001_00_000001, Exiting
2016-05-17 22:37:43,323 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524663078_0001_00_000001
2016-05-17 22:37:43,324 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524663078_0001_00_000001
2016-05-17 22:37:43,325 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524663078_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:43,325 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524663078_0001_00_000001 completed successfully
2016-05-17 22:37:43,326 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524663078_0001_1, queueSize=0
2016-05-17 22:37:43,327 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:43,328 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663078_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524663078_0001_1, startTime=1463524663239, finishTime=1463524663325, timeTaken=86, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524663305, Process_Duration=4, Process_End_Time=1463524663309, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:43,328 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524663078_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:43,328 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524663078_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:43,328 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524663078_0001_00_000001
2016-05-17 22:37:43,329 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663078_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:43 Completed Dag: dag_1463524663078_0001_1
2016-05-17 22:37:43,330 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524663078_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:43,330 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524663078_0001_1_post
2016-05-17 22:37:43,330 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:43,331 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:43,335 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...SULTS.0_'field2'_'field2', applicationId=application_1463524663078_0001
2016-05-17 22:37:43,337 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:43,337 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:43,337 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:43,337 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:43,337 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:43,338 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:43,338 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:43,338 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:43,338 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:43,339 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:43,339 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:43,339 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:43,339 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:43,343 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/B0B7FE22B0AE463A9022BFCEA6785A70/F439AE461D0645DEA7503537DD310C50/.tez/application_1463524663078_0001
2016-05-17 22:37:43,343 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_'field2'/_temporary
2016-05-17 22:37:43,344 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.282
2016-05-17 22:37:43,410 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_'field2'
2016-05-17 22:37:43,430 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:43,430 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:43,465 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 22:37:43,470 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 22:37:43,471 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:43,471 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:43,471 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:43,485 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:43,486 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS
2016-05-17 22:37:43,495 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:43,496 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:43,496 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS"]
2016-05-17 22:37:43,497 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:43,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:43,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:43,499 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-17 22:37:43,499 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:43,512 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:43,517 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:43,518 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:43,519 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:43,529 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/956DE4B768B740ADBDB725EC51E4DA55/D11CB534D1254D388D4B6FA9D9688B1B/.tez/application_1463524663518_0001 doesn't exist and is created
2016-05-17 22:37:43,545 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:43,545 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:43,545 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/956DE4B768B740ADBDB725EC51E4DA55/D11CB534D1254D388D4B6FA9D9688B1B/.tez/application_1463524663518_0001
2016-05-17 22:37:43,548 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524663518_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:43,558 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:43,559 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:43,561 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:43,561 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:43,561 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/956DE4B768B740ADBDB725EC51E4DA55/D11CB534D1254D388D4B6FA9D9688B1B/.tez/application_1463524663518_0001/localmode-log-dir
2016-05-17 22:37:43,562 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/956DE4B768B740ADBDB725EC51E4DA55/D11CB534D1254D388D4B6FA9D9688B1B/.tez/application_1463524663518_0001/localmode-log-dir/history.txt.appattempt_1463524663518_0001_000000, maxErrors=10
2016-05-17 22:37:43,562 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:43,562 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524663518_0001_000000, appSubmitTime=1463524663547, launchTime=1463524663548
2016-05-17 22:37:43,566 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:43,573 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:43,574 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:43,574 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:43,575 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41902
2016-05-17 22:37:43,576 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:43,579 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524663518_0001_000000, startTime=1463524663579
2016-05-17 22:37:43,579 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:43,579 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:43,640 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:43,640 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:43,645 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:43,645 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:43,645 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ESULTS.0_'field2'_RESULTS, applicationId=application_1463524663518_0001, dagName=[956DE4B768B740ADBDB725EC51E4DA55/D11CB534D1254D388D4B6FA9D9688B1B] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-17 22:37:43,661 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [956DE4B768B740ADBDB725EC51E4DA55/D11CB534D1254D388D4B6FA9D9688B1B] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-17 22:37:43,662 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524663518_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/956DE4B768B740ADBDB725EC51E4DA55/D11CB534D1254D388D4B6FA9D9688B1B/.tez/application_1463524663518_0001/localmode-log-dir/dag_1463524663518_0001_1.dot
2016-05-17 22:37:43,663 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/956DE4B768B740ADBDB725EC51E4DA55/D11CB534D1254D388D4B6FA9D9688B1B/.tez/application_1463524663518_0001/localmode-log-dir/dag_1463524663518_0001_1-tez-dag.pb.txt
2016-05-17 22:37:43,676 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524663518_0001_1
2016-05-17 22:37:43,676 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:43,676 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [956DE4B768B740ADBDB725EC51E4DA55/D11CB534D1254D388D4B6FA9D9688B1B] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-17 22:37:43 Running Dag: dag_1463524663518_0001_1
2016-05-17 22:37:43,689 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663518_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524663518_0001_1, submitTime=1463524663661
2016-05-17 22:37:43,689 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:43,690 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DBEE8CAC01EB41279C027ACA128AA2E4
2016-05-17 22:37:43,690 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:43,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663518_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524663518_0001_1, initTime=1463524663690
2016-05-17 22:37:43,691 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524663518_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:43,691 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:43,692 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663518_0001_1][Event:DAG_STARTED]: dagID=dag_1463524663518_0001_1, startTime=1463524663692
2016-05-17 22:37:43,692 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:43,692 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524663518_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:43,692 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DBEE8CAC01EB41279C027ACA128AA2E4 : {EAD24B534D224AE2A020E76290EFA3CF={InputName=EAD24B534D224AE2A020E76290EFA3CF}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:43,693 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: EAD24B534D224AE2A020E76290EFA3CF, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:43,692 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:43,693 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4]
2016-05-17 22:37:43,694 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4]
2016-05-17 22:37:43,694 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4]
2016-05-17 22:37:43,694 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4]. Starting root input initializers: 1
2016-05-17 22:37:43,696 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:43,696 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: EAD24B534D224AE2A020E76290EFA3CF on vertex vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4]
2016-05-17 22:37:43,697 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ESULTS.0_'field2'_RESULTS, applicationId=application_1463524663518_0001, dagName=[956DE4B768B740ADBDB725EC51E4DA55/D11CB534D1254D388D4B6FA9D9688B1B] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS
2016-05-17 22:37:43,699 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:43,699 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input EAD24B534D224AE2A020E76290EFA3CF asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:43,699 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:43,700 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:43,701 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:43,701 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: EAD24B534D224AE2A020E76290EFA3CF on vertex vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4]
2016-05-17 22:37:43,701 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:43,701 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {EAD24B534D224AE2A020E76290EFA3CF=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:43,702 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4] parallelism set to 1
2016-05-17 22:37:43,702 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:43,702 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663518_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524663518_0001_1_00, eventCount=1
2016-05-17 22:37:43,702 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4]
2016-05-17 22:37:43,702 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A8FF346CA1E34ABA9B6C2B076A99B924, vertexId=vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:43,703 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A8FF346CA1E34ABA9B6C2B076A99B924, vertexId=vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4]
2016-05-17 22:37:43,704 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A8FF346CA1E34ABA9B6C2B076A99B924, vertexName=DBEE8CAC01EB41279C027ACA128AA2E4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:43,704 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A8FF346CA1E34ABA9B6C2B076A99B924, vertexId=vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4]
2016-05-17 22:37:43,705 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663518_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DBEE8CAC01EB41279C027ACA128AA2E4, vertexId=vertex_1463524663518_0001_1_00, initRequestedTime=1463524663692, initedTime=1463524663705, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:43,706 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4] with distanceFromRoot: 0
2016-05-17 22:37:43,705 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:43,706 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:43,706 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DBEE8CAC01EB41279C027ACA128AA2E4
2016-05-17 22:37:43,706 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4]
2016-05-17 22:37:43,707 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:43,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663518_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524663518_0001_1_00, startRequestedTime=1463524663706, startedTime=1463524663706
2016-05-17 22:37:43,707 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:43,707 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:43,708 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663518_0001_1][Event:TASK_STARTED]: vertexName=DBEE8CAC01EB41279C027ACA128AA2E4, taskId=task_1463524663518_0001_1_00_000000, scheduledTime=1463524663707, launchTime=1463524663707
2016-05-17 22:37:43,708 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524663518_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:43,708 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4]
2016-05-17 22:37:43,708 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [956DE4B768B740ADBDB725EC51E4DA55/D11CB534D1254D388D4B6FA9D9688B1B] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS, VertexName: DBEE8CAC01EB41279C027ACA128AA2E4, VertexParallelism: 1, TaskAttemptID:attempt_1463524663518_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EAD24B534D224AE2A020E76290EFA3CF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A8FF346CA1E34ABA9B6C2B076A99B924, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:43,709 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524663518_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:43,709 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:43,710 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:43,711 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:43,711 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:43,711 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524663518_0001_00_000001
2016-05-17 22:37:43,711 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663518_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:43,713 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524663518_0001_00_000001
2016-05-17 22:37:43,713 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:43,714 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663518_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:43,714 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:43,714 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663518_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524663518_0001_00_000001, launchTime=1463524663713
2016-05-17 22:37:43,715 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524663518_0001_00_000001
2016-05-17 22:37:43,716 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524663518_0001_1_00_000000_0] to container: [container_1463524663518_0001_00_000001]
2016-05-17 22:37:43,716 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663518_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:43,716 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524663518_0001, with dag id: dag_1463524663518_0001_1
2016-05-17 22:37:43,717 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524663518_0001_1_00_000000_0] started. Is using containerId: [container_1463524663518_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:43,718 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663518_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DBEE8CAC01EB41279C027ACA128AA2E4, taskAttemptId=attempt_1463524663518_0001_1_00_000000_0, startTime=1463524663717, containerId=container_1463524663518_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524663518_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:43,718 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524663518_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:43,718 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524663518_0001_00_000001 given task: attempt_1463524663518_0001_1_00_000000_0
2016-05-17 22:37:43,718 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524663518_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524663518_0001_1_00_000000_0
2016-05-17 22:37:43,718 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:43,718 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524663518_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:43 Starting to run new task attempt: attempt_1463524663518_0001_1_00_000000_0
2016-05-17 22:37:43,719 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524663518_0001_1_00_000000_0
2016-05-17 22:37:43,719 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:43,720 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:43,720 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:43,720 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:43,720 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:43,720 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:43,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [956DE4B768B740ADBDB725EC51E4DA55/D11CB534D1254D388D4B6FA9D9688B1B] every.every/(1/1) ...ESULTS.0_'field2'_RESULTS, VertexName: DBEE8CAC01EB41279C027ACA128AA2E4, VertexParallelism: 1, TaskAttemptID:attempt_1463524663518_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EAD24B534D224AE2A020E76290EFA3CF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A8FF346CA1E34ABA9B6C2B076A99B924, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:43,720 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:43,722 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524663518_0001_1_00_000000_0
2016-05-17 22:37:43,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:43,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:43,724 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EAD24B534D224AE2A020E76290EFA3CF, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:43,724 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:43,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EAD24B534D224AE2A020E76290EFA3CF
2016-05-17 22:37:43,725 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: EAD24B534D224AE2A020E76290EFA3CF
2016-05-17 22:37:43,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:43,727 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524663518_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:43,728 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:43,728 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A8FF346CA1E34ABA9B6C2B076A99B924, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:43,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:43,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A8FF346CA1E34ABA9B6C2B076A99B924
2016-05-17 22:37:43,731 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:43,732 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A8FF346CA1E34ABA9B6C2B076A99B924
2016-05-17 22:37:43,730 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:43,732 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: EAD24B534D224AE2A020E76290EFA3CF
2016-05-17 22:37:43,732 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EAD24B534D224AE2A020E76290EFA3CF
2016-05-17 22:37:43,737 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DBEE8CAC01EB41279C027ACA128AA2E4, ordinal: 0
2016-05-17 22:37:43,737 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DBEE8CAC01EB41279C027ACA128AA2E4, mem on start (mb), free: 154, total: 385, max: 672
2016-05-17 22:37:43,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:43,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:43,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:43,741 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:43,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A8FF346CA1E34ABA9B6C2B076A99B924
2016-05-17 22:37:43,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:43,742 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:43,742 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:43,743 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:43,743 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:43,743 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:43,744 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:43,744 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:43,744 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EAD24B534D224AE2A020E76290EFA3CF, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:43,744 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A8FF346CA1E34ABA9B6C2B076A99B924, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:43,745 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:43,745 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EAD24B534D224AE2A020E76290EFA3CF
2016-05-17 22:37:43,745 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:43,745 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EAD24B534D224AE2A020E76290EFA3CF
2016-05-17 22:37:43,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EAD24B534D224AE2A020E76290EFA3CF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:43,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:43,747 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:43,747 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:43,747 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524663518_0001_1_00_000000_0
2016-05-17 22:37:43,750 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:43,751 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: EAD24B534D224AE2A020E76290EFA3CF
2016-05-17 22:37:43,751 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS"], id: A8FF346CA1E34ABA9B6C2B076A99B924
2016-05-17 22:37:43,751 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS"]
2016-05-17 22:37:43,751 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:43,751 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:43,752 INFO input.MRInput (MRInput.java:checkAndAwaitRecordReaderInitialization(597)) - Awaiting RecordReader initialization
2016-05-17 22:37:43,752 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:43,753 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4cd8b18
2016-05-17 22:37:43,754 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4cd8b18
2016-05-17 22:37:43,754 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:43,755 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:43,755 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DBEE8CAC01EB41279C027ACA128AA2E4, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:43,757 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:43,757 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:43,758 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524663518_0001_1_00_000000_0
2016-05-17 22:37:43,758 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524663518_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:43,759 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524663518_0001_1_00_000000_0
2016-05-17 22:37:43,759 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DBEE8CAC01EB41279C027ACA128AA2E4, mem on close (mb), free: 153, total: 385, max: 672
2016-05-17 22:37:43 Completed running task attempt: attempt_1463524663518_0001_1_00_000000_0
2016-05-17 22:37:43,761 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524663518_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:43,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524663755, Process_Duration=4, Process_End_Time=1463524663759, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:43,762 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:43,763 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:43,763 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663518_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DBEE8CAC01EB41279C027ACA128AA2E4, taskAttemptId=attempt_1463524663518_0001_1_00_000000_0, startTime=1463524663717, finishTime=1463524663762, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524663755, Process_Duration=4, Process_End_Time=1463524663759, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:43,763 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:43,763 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524663518_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:43,763 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:43,763 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524663518_0001_1_00_000000_0
2016-05-17 22:37:43,764 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524663518_0001_00_000001
2016-05-17 22:37:43,764 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:43,764 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663518_0001_1][Event:TASK_FINISHED]: vertexName=DBEE8CAC01EB41279C027ACA128AA2E4, taskId=task_1463524663518_0001_1_00_000000, startTime=1463524663717, finishTime=1463524663764, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1463524663518_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524663755, Process_Duration=4, Process_End_Time=1463524663759, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:43,765 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524663518_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:43,765 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4] : 1
2016-05-17 22:37:43,765 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4]
2016-05-17 22:37:43,766 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:43,767 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663518_0001_1][Event:VERTEX_FINISHED]: vertexName=DBEE8CAC01EB41279C027ACA128AA2E4, vertexId=vertex_1463524663518_0001_1_00, initRequestedTime=1463524663692, initedTime=1463524663705, startRequestedTime=1463524663706, startedTime=1463524663706, finishTime=1463524663765, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524663755, Process_Duration=4, Process_End_Time=1463524663759, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524663717, firstTasksToStart=[ task_1463524663518_0001_1_00_000000 ], lastTaskFinishTime=1463524663762, lastTasksToFinish=[ task_1463524663518_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524663518_0001_1_00_000000 ], longestDurationTasks=[ task_1463524663518_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:43,767 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:43,768 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663518_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:43,768 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:43,769 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663518_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524663518_0001_00_000001, stoppedTime=1463524663768, exitStatus=0
2016-05-17 22:37:43,769 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663518_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:43,769 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:43,769 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524663518_0001_1
2016-05-17 22:37:43,770 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663518_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524663518_0001_1
2016-05-17 22:37:43,770 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A8FF346CA1E34ABA9B6C2B076A99B924 for vertex: vertex_1463524663518_0001_1_00 [DBEE8CAC01EB41279C027ACA128AA2E4]
2016-05-17 22:37:43,774 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524663518_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:43,775 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524663518_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:43,775 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524663518_0001_00_000001, Exiting
2016-05-17 22:37:43,775 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524663518_0001_00_000001
2016-05-17 22:37:43,775 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524663518_0001_00_000001
2016-05-17 22:37:43,776 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524663518_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:43,776 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524663518_0001_00_000001 completed successfully
2016-05-17 22:37:43,780 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524663518_0001_1, queueSize=0
2016-05-17 22:37:43,781 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:43,781 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663518_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524663518_0001_1, startTime=1463524663692, finishTime=1463524663779, timeTaken=87, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404226048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524663755, Process_Duration=4, Process_End_Time=1463524663759, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:43,782 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524663518_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:43,782 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524663518_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:43,782 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663518_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:43,782 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524663518_0001_00_000001
2016-05-17 22:37:43 Completed Dag: dag_1463524663518_0001_1
2016-05-17 22:37:43,783 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524663518_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:43,784 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524663518_0001_1_post
2016-05-17 22:37:43,784 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:43,784 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:43,789 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ESULTS.0_'field2'_RESULTS, applicationId=application_1463524663518_0001
2016-05-17 22:37:43,790 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:43,791 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:43,791 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:43,791 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:43,791 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:43,792 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:43,792 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:43,792 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:43,793 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:43,793 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:43,793 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:43,793 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:43,794 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:43,797 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/956DE4B768B740ADBDB725EC51E4DA55/D11CB534D1254D388D4B6FA9D9688B1B/.tez/application_1463524663518_0001
2016-05-17 22:37:43,798 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_RESULTS/_temporary
2016-05-17 22:37:43,798 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.303
2016-05-17 22:37:43,878 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_RESULTS
2016-05-17 22:37:43,902 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:43,902 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:43,939 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 22:37:43,950 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-17 22:37:43,950 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:43,950 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:43,951 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:43,967 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:43,968 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL
2016-05-17 22:37:43,977 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:43,977 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:43,977 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL"]
2016-05-17 22:37:43,977 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:43,977 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:43,977 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:43,978 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-17 22:37:43,978 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:43,990 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:43,994 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:43,995 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:43,995 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:44,004 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66B46CA10CEA4B4BA9D135208496B42B/F9AF24FE4F5B49FB9506A454BB7AB247/.tez/application_1463524663995_0001 doesn't exist and is created
2016-05-17 22:37:44,021 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:44,021 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:44,021 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66B46CA10CEA4B4BA9D135208496B42B/F9AF24FE4F5B49FB9506A454BB7AB247/.tez/application_1463524663995_0001
2016-05-17 22:37:44,024 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524663995_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:44,034 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:44,034 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:44,036 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:44,036 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:44,036 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66B46CA10CEA4B4BA9D135208496B42B/F9AF24FE4F5B49FB9506A454BB7AB247/.tez/application_1463524663995_0001/localmode-log-dir
2016-05-17 22:37:44,036 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66B46CA10CEA4B4BA9D135208496B42B/F9AF24FE4F5B49FB9506A454BB7AB247/.tez/application_1463524663995_0001/localmode-log-dir/history.txt.appattempt_1463524663995_0001_000000, maxErrors=10
2016-05-17 22:37:44,037 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:44,037 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524663995_0001_000000, appSubmitTime=1463524664024, launchTime=1463524664024
2016-05-17 22:37:44,041 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:44,042 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:44,043 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:44,044 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:38461
2016-05-17 22:37:44,046 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:44,049 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:44,051 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524663995_0001_000000, startTime=1463524664051
2016-05-17 22:37:44,051 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:44,051 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:44,121 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:44,121 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:44,121 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_'field2'_ALL, applicationId=application_1463524663995_0001, dagName=[66B46CA10CEA4B4BA9D135208496B42B/F9AF24FE4F5B49FB9506A454BB7AB247] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-17 22:37:44,137 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [66B46CA10CEA4B4BA9D135208496B42B/F9AF24FE4F5B49FB9506A454BB7AB247] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-17 22:37:44,139 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524663995_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66B46CA10CEA4B4BA9D135208496B42B/F9AF24FE4F5B49FB9506A454BB7AB247/.tez/application_1463524663995_0001/localmode-log-dir/dag_1463524663995_0001_1.dot
2016-05-17 22:37:44,140 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66B46CA10CEA4B4BA9D135208496B42B/F9AF24FE4F5B49FB9506A454BB7AB247/.tez/application_1463524663995_0001/localmode-log-dir/dag_1463524663995_0001_1-tez-dag.pb.txt
2016-05-17 22:37:44,148 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524663995_0001_1
2016-05-17 22:37:44,148 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:44,148 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [66B46CA10CEA4B4BA9D135208496B42B/F9AF24FE4F5B49FB9506A454BB7AB247] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-17 22:37:44 Running Dag: dag_1463524663995_0001_1
2016-05-17 22:37:44,164 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:44,164 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663995_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524663995_0001_1, submitTime=1463524664137
2016-05-17 22:37:44,165 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex ACDF13530FB44FD683402449B5BE3C06
2016-05-17 22:37:44,165 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:44,165 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663995_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524663995_0001_1, initTime=1463524664164
2016-05-17 22:37:44,165 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:44,166 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524663995_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:44,167 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663995_0001_1][Event:DAG_STARTED]: dagID=dag_1463524663995_0001_1, startTime=1463524664166
2016-05-17 22:37:44,167 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:44,167 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:44,167 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524663995_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:44,167 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: ACDF13530FB44FD683402449B5BE3C06 : {69AF021EA4644C3290F4B606E2A83663={InputName=69AF021EA4644C3290F4B606E2A83663}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:44,168 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 69AF021EA4644C3290F4B606E2A83663, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:44,168 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06]
2016-05-17 22:37:44,168 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06]
2016-05-17 22:37:44,168 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06]
2016-05-17 22:37:44,168 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06]. Starting root input initializers: 1
2016-05-17 22:37:44,169 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:44,169 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_'field2'_ALL, applicationId=application_1463524663995_0001, dagName=[66B46CA10CEA4B4BA9D135208496B42B/F9AF24FE4F5B49FB9506A454BB7AB247] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL
2016-05-17 22:37:44,169 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 69AF021EA4644C3290F4B606E2A83663 on vertex vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06]
2016-05-17 22:37:44,171 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:44,172 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 69AF021EA4644C3290F4B606E2A83663 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:44,172 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:44,172 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:44,174 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:44,174 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 69AF021EA4644C3290F4B606E2A83663 on vertex vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06]
2016-05-17 22:37:44,175 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:44,175 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {69AF021EA4644C3290F4B606E2A83663=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:44,175 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06] parallelism set to 1
2016-05-17 22:37:44,175 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:44,175 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663995_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524663995_0001_1_00, eventCount=1
2016-05-17 22:37:44,176 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06]
2016-05-17 22:37:44,176 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=05E6377F99C84717800C7EA53D66821E, vertexId=vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:44,176 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=05E6377F99C84717800C7EA53D66821E, vertexId=vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06]
2016-05-17 22:37:44,178 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=05E6377F99C84717800C7EA53D66821E, vertexName=ACDF13530FB44FD683402449B5BE3C06, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:44,178 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=05E6377F99C84717800C7EA53D66821E, vertexId=vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06]
2016-05-17 22:37:44,180 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:44,180 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663995_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ACDF13530FB44FD683402449B5BE3C06, vertexId=vertex_1463524663995_0001_1_00, initRequestedTime=1463524664167, initedTime=1463524664180, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:44,181 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06] with distanceFromRoot: 0
2016-05-17 22:37:44,181 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:44,182 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in ACDF13530FB44FD683402449B5BE3C06
2016-05-17 22:37:44,182 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06]
2016-05-17 22:37:44,182 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:44,182 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663995_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524663995_0001_1_00, startRequestedTime=1463524664181, startedTime=1463524664181
2016-05-17 22:37:44,183 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:44,183 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663995_0001_1][Event:TASK_STARTED]: vertexName=ACDF13530FB44FD683402449B5BE3C06, taskId=task_1463524663995_0001_1_00_000000, scheduledTime=1463524664183, launchTime=1463524664183
2016-05-17 22:37:44,183 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:44,184 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524663995_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:44,184 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06]
2016-05-17 22:37:44,184 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [66B46CA10CEA4B4BA9D135208496B42B/F9AF24FE4F5B49FB9506A454BB7AB247] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL, VertexName: ACDF13530FB44FD683402449B5BE3C06, VertexParallelism: 1, TaskAttemptID:attempt_1463524663995_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=69AF021EA4644C3290F4B606E2A83663, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=05E6377F99C84717800C7EA53D66821E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:44,185 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524663995_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:44,185 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:44,187 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524663995_0001, with dag id: dag_1463524663995_0001_1
2016-05-17 22:37:44,191 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:44,193 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:44,193 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:44,193 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524663995_0001_00_000001
2016-05-17 22:37:44,194 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663995_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:44,195 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524663995_0001_00_000001
2016-05-17 22:37:44,195 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:44,196 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663995_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:44,197 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:44,197 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663995_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524663995_0001_00_000001, launchTime=1463524664197
2016-05-17 22:37:44,198 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524663995_0001_00_000001
2016-05-17 22:37:44,199 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524663995_0001_1_00_000000_0] to container: [container_1463524663995_0001_00_000001]
2016-05-17 22:37:44,199 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663995_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:44,200 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524663995_0001_1_00_000000_0] started. Is using containerId: [container_1463524663995_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:44,200 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:44,200 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663995_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ACDF13530FB44FD683402449B5BE3C06, taskAttemptId=attempt_1463524663995_0001_1_00_000000_0, startTime=1463524664200, containerId=container_1463524663995_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524663995_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:44,200 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524663995_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:44,200 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524663995_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:44,201 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524663995_0001_00_000001 given task: attempt_1463524663995_0001_1_00_000000_0
2016-05-17 22:37:44,201 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524663995_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524663995_0001_1_00_000000_0
2016-05-17 22:37:44 Starting to run new task attempt: attempt_1463524663995_0001_1_00_000000_0
2016-05-17 22:37:44,202 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524663995_0001_1_00_000000_0
2016-05-17 22:37:44,202 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:44,202 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:44,202 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:44,202 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:44,202 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:44,203 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:44,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [66B46CA10CEA4B4BA9D135208496B42B/F9AF24FE4F5B49FB9506A454BB7AB247] every.every/(1/1) ...d'_RESULTS.0_'field2'_ALL, VertexName: ACDF13530FB44FD683402449B5BE3C06, VertexParallelism: 1, TaskAttemptID:attempt_1463524663995_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=69AF021EA4644C3290F4B606E2A83663, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=05E6377F99C84717800C7EA53D66821E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:44,204 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:44,205 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524663995_0001_1_00_000000_0
2016-05-17 22:37:44,205 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:44,205 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:44,206 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=69AF021EA4644C3290F4B606E2A83663, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:44,206 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:44,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 69AF021EA4644C3290F4B606E2A83663
2016-05-17 22:37:44,208 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 69AF021EA4644C3290F4B606E2A83663
2016-05-17 22:37:44,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:44,212 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524663995_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:44,213 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:44,213 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 69AF021EA4644C3290F4B606E2A83663
2016-05-17 22:37:44,213 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 69AF021EA4644C3290F4B606E2A83663
2016-05-17 22:37:44,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=05E6377F99C84717800C7EA53D66821E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:44,214 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:44,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:44,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 05E6377F99C84717800C7EA53D66821E
2016-05-17 22:37:44,218 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:44,218 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 05E6377F99C84717800C7EA53D66821E
2016-05-17 22:37:44,225 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ACDF13530FB44FD683402449B5BE3C06, ordinal: 0
2016-05-17 22:37:44,225 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ACDF13530FB44FD683402449B5BE3C06, mem on start (mb), free: 281, total: 386, max: 672
2016-05-17 22:37:44,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:44,226 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:44,226 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:44,231 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:44,232 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 05E6377F99C84717800C7EA53D66821E
2016-05-17 22:37:44,232 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:44,232 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:44,232 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:44,233 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:44,233 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:44,233 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:44,233 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:44,233 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:44,233 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 69AF021EA4644C3290F4B606E2A83663, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:44,233 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 05E6377F99C84717800C7EA53D66821E, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:44,233 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:44,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 69AF021EA4644C3290F4B606E2A83663 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:44,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:44,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 69AF021EA4644C3290F4B606E2A83663
2016-05-17 22:37:44,234 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:44,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 69AF021EA4644C3290F4B606E2A83663
2016-05-17 22:37:44,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:44,235 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:44,236 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524663995_0001_1_00_000000_0
2016-05-17 22:37:44,239 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:44,240 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:44,240 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 69AF021EA4644C3290F4B606E2A83663
2016-05-17 22:37:44,241 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL"], id: 05E6377F99C84717800C7EA53D66821E
2016-05-17 22:37:44,241 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL"]
2016-05-17 22:37:44,242 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:44,242 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:44,242 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@61e04742
2016-05-17 22:37:44,243 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@61e04742
2016-05-17 22:37:44,243 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:44,245 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:44,245 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ACDF13530FB44FD683402449B5BE3C06, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:44,249 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:44,249 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:44,250 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524663995_0001_1_00_000000_0
2016-05-17 22:37:44,250 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524663995_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:44,250 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524663995_0001_1_00_000000_0
2016-05-17 22:37:44,251 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ACDF13530FB44FD683402449B5BE3C06, mem on close (mb), free: 280, total: 386, max: 672
2016-05-17 22:37:44 Completed running task attempt: attempt_1463524663995_0001_1_00_000000_0
2016-05-17 22:37:44,253 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524663995_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:44,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:44,255 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663995_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ACDF13530FB44FD683402449B5BE3C06, taskAttemptId=attempt_1463524663995_0001_1_00_000000_0, startTime=1463524664200, finishTime=1463524664254, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524664245, Process_Duration=5, Process_End_Time=1463524664250, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:44,255 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524663995_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:44,255 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524663995_0001_1_00_000000_0
2016-05-17 22:37:44,255 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:44,256 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:44,256 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663995_0001_1][Event:TASK_FINISHED]: vertexName=ACDF13530FB44FD683402449B5BE3C06, taskId=task_1463524663995_0001_1_00_000000, startTime=1463524664200, finishTime=1463524664255, timeTaken=55, status=SUCCEEDED, successfulAttemptID=attempt_1463524663995_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524664245, Process_Duration=5, Process_End_Time=1463524664250, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:44,256 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524663995_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:44,256 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06] : 1
2016-05-17 22:37:44,256 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06]
2016-05-17 22:37:44,257 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:44,258 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663995_0001_1][Event:VERTEX_FINISHED]: vertexName=ACDF13530FB44FD683402449B5BE3C06, vertexId=vertex_1463524663995_0001_1_00, initRequestedTime=1463524664167, initedTime=1463524664180, startRequestedTime=1463524664181, startedTime=1463524664181, finishTime=1463524664256, timeTaken=75, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524664245, Process_Duration=5, Process_End_Time=1463524664250, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524664200, firstTasksToStart=[ task_1463524663995_0001_1_00_000000 ], lastTaskFinishTime=1463524664254, lastTasksToFinish=[ task_1463524663995_0001_1_00_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524663995_0001_1_00_000000 ], longestDurationTasks=[ task_1463524663995_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:44,258 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:44,259 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663995_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:44,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663995_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524663995_0001_00_000001, stoppedTime=1463524664259, exitStatus=0
2016-05-17 22:37:44,259 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663995_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:44,259 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:44,259 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524663995_0001_1
2016-05-17 22:37:44,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663995_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524663995_0001_1
2016-05-17 22:37:44,259 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:44,259 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 05E6377F99C84717800C7EA53D66821E for vertex: vertex_1463524663995_0001_1_00 [ACDF13530FB44FD683402449B5BE3C06]
2016-05-17 22:37:44,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524664245, Process_Duration=5, Process_End_Time=1463524664250, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:44,260 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:44,260 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:44,260 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524663995_0001_00_000001
2016-05-17 22:37:44,261 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524663995_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:44,261 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524663995_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:44,261 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524663995_0001_00_000001, Exiting
2016-05-17 22:37:44,261 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524663995_0001_00_000001
2016-05-17 22:37:44,261 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524663995_0001_00_000001
2016-05-17 22:37:44,264 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524663995_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:44,265 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524663995_0001_00_000001 completed successfully
2016-05-17 22:37:44,269 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524663995_0001_1, queueSize=0
2016-05-17 22:37:44,269 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:44,269 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524663995_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524663995_0001_1, startTime=1463524664166, finishTime=1463524664268, timeTaken=102, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524664245, Process_Duration=5, Process_End_Time=1463524664250, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:44,270 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524663995_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:44,270 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524663995_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:44,270 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524663995_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:44,270 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524663995_0001_00_000001
2016-05-17 22:37:44 Completed Dag: dag_1463524663995_0001_1
2016-05-17 22:37:44,270 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524663995_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:44,270 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524663995_0001_1_post
2016-05-17 22:37:44,271 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:44,271 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:44,277 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.0_'field2'_ALL, applicationId=application_1463524663995_0001
2016-05-17 22:37:44,279 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:44,279 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:44,279 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:44,279 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:44,279 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:44,280 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:44,280 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:44,281 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:44,282 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:44,282 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:44,282 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:44,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:44,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:44,287 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/66B46CA10CEA4B4BA9D135208496B42B/F9AF24FE4F5B49FB9506A454BB7AB247/.tez/application_1463524663995_0001
2016-05-17 22:37:44,287 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_'field2'_ALL/_temporary
2016-05-17 22:37:44,288 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.310
2016-05-17 22:37:44,342 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:44,342 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:44,369 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_'field2'_ALL
2016-05-17 22:37:44,390 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:44,390 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:44,427 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 22:37:44,427 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 22:37:44,428 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:44,428 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:44,428 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:44,442 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:44,442 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1
2016-05-17 22:37:44,451 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:44,452 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:44,452 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1"]
2016-05-17 22:37:44,452 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:44,453 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:44,453 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:44,454 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-17 22:37:44,454 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:44,468 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:44,473 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:44,473 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:44,473 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:44,483 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BF535445E13A49299B0BA4E74B00B602/DF815BE1896B45CFBC7C8A38BFA6B942/.tez/application_1463524664473_0001 doesn't exist and is created
2016-05-17 22:37:44,498 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:44,499 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:44,499 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BF535445E13A49299B0BA4E74B00B602/DF815BE1896B45CFBC7C8A38BFA6B942/.tez/application_1463524664473_0001
2016-05-17 22:37:44,502 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524664473_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:44,514 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:44,514 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:44,516 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:44,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:44,516 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BF535445E13A49299B0BA4E74B00B602/DF815BE1896B45CFBC7C8A38BFA6B942/.tez/application_1463524664473_0001/localmode-log-dir
2016-05-17 22:37:44,517 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BF535445E13A49299B0BA4E74B00B602/DF815BE1896B45CFBC7C8A38BFA6B942/.tez/application_1463524664473_0001/localmode-log-dir/history.txt.appattempt_1463524664473_0001_000000, maxErrors=10
2016-05-17 22:37:44,517 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:44,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524664473_0001_000000, appSubmitTime=1463524664502, launchTime=1463524664502
2016-05-17 22:37:44,522 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:44,524 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:44,525 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:44,525 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:44,529 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:60370
2016-05-17 22:37:44,531 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:44,532 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524664473_0001_000000, startTime=1463524664532
2016-05-17 22:37:44,532 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:44,533 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:44,599 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:44,599 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:44,599 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, applicationId=application_1463524664473_0001, dagName=[BF535445E13A49299B0BA4E74B00B602/DF815BE1896B45CFBC7C8A38BFA6B942] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-17 22:37:44,616 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [BF535445E13A49299B0BA4E74B00B602/DF815BE1896B45CFBC7C8A38BFA6B942] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-17 22:37:44,617 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524664473_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BF535445E13A49299B0BA4E74B00B602/DF815BE1896B45CFBC7C8A38BFA6B942/.tez/application_1463524664473_0001/localmode-log-dir/dag_1463524664473_0001_1.dot
2016-05-17 22:37:44,617 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BF535445E13A49299B0BA4E74B00B602/DF815BE1896B45CFBC7C8A38BFA6B942/.tez/application_1463524664473_0001/localmode-log-dir/dag_1463524664473_0001_1-tez-dag.pb.txt
2016-05-17 22:37:44,623 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524664473_0001_1
2016-05-17 22:37:44,623 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:44,624 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [BF535445E13A49299B0BA4E74B00B602/DF815BE1896B45CFBC7C8A38BFA6B942] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-17 22:37:44 Running Dag: dag_1463524664473_0001_1
2016-05-17 22:37:44,637 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664473_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524664473_0001_1, submitTime=1463524664616
2016-05-17 22:37:44,637 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:44,639 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D6C2275D9CB74797880B41D79EA5B73B
2016-05-17 22:37:44,639 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:44,639 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664473_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524664473_0001_1, initTime=1463524664637
2016-05-17 22:37:44,640 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:44,641 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524664473_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:44,641 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664473_0001_1][Event:DAG_STARTED]: dagID=dag_1463524664473_0001_1, startTime=1463524664641
2016-05-17 22:37:44,641 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:44,641 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:44,641 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524664473_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:44,642 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D6C2275D9CB74797880B41D79EA5B73B : {0AA630D23A8649F5867E1892FDCC7284={InputName=0AA630D23A8649F5867E1892FDCC7284}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:44,642 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0AA630D23A8649F5867E1892FDCC7284, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:44,643 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B]
2016-05-17 22:37:44,643 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B]
2016-05-17 22:37:44,643 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B]
2016-05-17 22:37:44,643 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, applicationId=application_1463524664473_0001, dagName=[BF535445E13A49299B0BA4E74B00B602/DF815BE1896B45CFBC7C8A38BFA6B942] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1
2016-05-17 22:37:44,643 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B]. Starting root input initializers: 1
2016-05-17 22:37:44,643 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:44,644 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0AA630D23A8649F5867E1892FDCC7284 on vertex vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B]
2016-05-17 22:37:44,646 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:44,646 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0AA630D23A8649F5867E1892FDCC7284 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:44,646 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:44,646 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:44,648 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:44,648 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0AA630D23A8649F5867E1892FDCC7284 on vertex vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B]
2016-05-17 22:37:44,648 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:44,648 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0AA630D23A8649F5867E1892FDCC7284=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:44,649 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B] parallelism set to 1
2016-05-17 22:37:44,649 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:44,649 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664473_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524664473_0001_1_00, eventCount=1
2016-05-17 22:37:44,649 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B]
2016-05-17 22:37:44,649 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0B5C61320AAB40BCBD546552B3FF9A2B, vertexId=vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:44,650 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0B5C61320AAB40BCBD546552B3FF9A2B, vertexId=vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B]
2016-05-17 22:37:44,652 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0B5C61320AAB40BCBD546552B3FF9A2B, vertexName=D6C2275D9CB74797880B41D79EA5B73B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:44,652 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0B5C61320AAB40BCBD546552B3FF9A2B, vertexId=vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B]
2016-05-17 22:37:44,653 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:44,655 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664473_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D6C2275D9CB74797880B41D79EA5B73B, vertexId=vertex_1463524664473_0001_1_00, initRequestedTime=1463524664642, initedTime=1463524664653, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:44,655 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B] with distanceFromRoot: 0
2016-05-17 22:37:44,655 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:44,655 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D6C2275D9CB74797880B41D79EA5B73B
2016-05-17 22:37:44,655 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B]
2016-05-17 22:37:44,656 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:44,657 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664473_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524664473_0001_1_00, startRequestedTime=1463524664655, startedTime=1463524664655
2016-05-17 22:37:44,657 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:44,658 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:44,658 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664473_0001_1][Event:TASK_STARTED]: vertexName=D6C2275D9CB74797880B41D79EA5B73B, taskId=task_1463524664473_0001_1_00_000000, scheduledTime=1463524664658, launchTime=1463524664658
2016-05-17 22:37:44,659 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524664473_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:44,659 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B]
2016-05-17 22:37:44,659 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BF535445E13A49299B0BA4E74B00B602/DF815BE1896B45CFBC7C8A38BFA6B942] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, VertexName: D6C2275D9CB74797880B41D79EA5B73B, VertexParallelism: 1, TaskAttemptID:attempt_1463524664473_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0AA630D23A8649F5867E1892FDCC7284, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0B5C61320AAB40BCBD546552B3FF9A2B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:44,660 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524664473_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:44,660 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:44,661 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:44,662 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:44,662 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:44,662 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524664473_0001_00_000001
2016-05-17 22:37:44,662 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524664473_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:44,664 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524664473_0001_00_000001
2016-05-17 22:37:44,664 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:44,665 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524664473_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:44,665 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:44,666 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664473_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524664473_0001_00_000001, launchTime=1463524664665
2016-05-17 22:37:44,666 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524664473_0001, with dag id: dag_1463524664473_0001_1
2016-05-17 22:37:44,667 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524664473_0001_00_000001
2016-05-17 22:37:44,668 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524664473_0001_1_00_000000_0] to container: [container_1463524664473_0001_00_000001]
2016-05-17 22:37:44,669 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524664473_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:44,669 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524664473_0001_1_00_000000_0] started. Is using containerId: [container_1463524664473_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:44,670 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664473_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D6C2275D9CB74797880B41D79EA5B73B, taskAttemptId=attempt_1463524664473_0001_1_00_000000_0, startTime=1463524664669, containerId=container_1463524664473_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524664473_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:44,670 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524664473_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:44,670 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:44,670 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524664473_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:44,670 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524664473_0001_00_000001 given task: attempt_1463524664473_0001_1_00_000000_0
2016-05-17 22:37:44,671 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524664473_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524664473_0001_1_00_000000_0
2016-05-17 22:37:44 Starting to run new task attempt: attempt_1463524664473_0001_1_00_000000_0
2016-05-17 22:37:44,672 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524664473_0001_1_00_000000_0
2016-05-17 22:37:44,672 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:44,672 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:44,672 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:44,672 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:44,672 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:44,672 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:44,672 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BF535445E13A49299B0BA4E74B00B602/DF815BE1896B45CFBC7C8A38BFA6B942] every.every/(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, VertexName: D6C2275D9CB74797880B41D79EA5B73B, VertexParallelism: 1, TaskAttemptID:attempt_1463524664473_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0AA630D23A8649F5867E1892FDCC7284, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0B5C61320AAB40BCBD546552B3FF9A2B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:44,673 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:44,673 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524664473_0001_1_00_000000_0
2016-05-17 22:37:44,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:44,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:44,674 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524664473_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:44,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:44,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0AA630D23A8649F5867E1892FDCC7284, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:44,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:44,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0AA630D23A8649F5867E1892FDCC7284
2016-05-17 22:37:44,675 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0AA630D23A8649F5867E1892FDCC7284
2016-05-17 22:37:44,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0B5C61320AAB40BCBD546552B3FF9A2B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:44,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:44,677 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:44,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0B5C61320AAB40BCBD546552B3FF9A2B
2016-05-17 22:37:44,677 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:44,678 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:44,678 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0B5C61320AAB40BCBD546552B3FF9A2B
2016-05-17 22:37:44,678 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0AA630D23A8649F5867E1892FDCC7284
2016-05-17 22:37:44,678 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0AA630D23A8649F5867E1892FDCC7284
2016-05-17 22:37:44,687 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D6C2275D9CB74797880B41D79EA5B73B, ordinal: 0
2016-05-17 22:37:44,687 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D6C2275D9CB74797880B41D79EA5B73B, mem on start (mb), free: 203, total: 386, max: 672
2016-05-17 22:37:44,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:44,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:44,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:44,689 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:44,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0B5C61320AAB40BCBD546552B3FF9A2B
2016-05-17 22:37:44,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:44,689 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:44,690 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:44,690 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:44,690 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:44,690 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:44,690 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:44,691 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:44,691 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0AA630D23A8649F5867E1892FDCC7284, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:44,691 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0B5C61320AAB40BCBD546552B3FF9A2B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:44,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:44,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0AA630D23A8649F5867E1892FDCC7284 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:44,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0AA630D23A8649F5867E1892FDCC7284
2016-05-17 22:37:44,691 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:44,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0AA630D23A8649F5867E1892FDCC7284
2016-05-17 22:37:44,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:44,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:44,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:44,693 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524664473_0001_1_00_000000_0
2016-05-17 22:37:44,693 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:44,693 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@31feced6
2016-05-17 22:37:44,694 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@31feced6
2016-05-17 22:37:44,694 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:44,695 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:44,696 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:44,697 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0AA630D23A8649F5867E1892FDCC7284
2016-05-17 22:37:44,697 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1"], id: 0B5C61320AAB40BCBD546552B3FF9A2B
2016-05-17 22:37:44,697 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1"]
2016-05-17 22:37:44,697 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:44,697 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:44,698 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D6C2275D9CB74797880B41D79EA5B73B, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:44,701 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:44,701 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:44,701 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524664473_0001_1_00_000000_0
2016-05-17 22:37:44,701 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524664473_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:44,702 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524664473_0001_1_00_000000_0
2016-05-17 22:37:44,702 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D6C2275D9CB74797880B41D79EA5B73B, mem on close (mb), free: 202, total: 386, max: 672
2016-05-17 22:37:44 Completed running task attempt: attempt_1463524664473_0001_1_00_000000_0
2016-05-17 22:37:44,703 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524664473_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:44,704 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664473_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D6C2275D9CB74797880B41D79EA5B73B, taskAttemptId=attempt_1463524664473_0001_1_00_000000_0, startTime=1463524664669, finishTime=1463524664704, timeTaken=35, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524664698, Process_Duration=4, Process_End_Time=1463524664702, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:44,704 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524664473_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:44,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:44,704 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:44,704 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524664473_0001_1_00_000000_0
2016-05-17 22:37:44,705 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:44,705 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664473_0001_1][Event:TASK_FINISHED]: vertexName=D6C2275D9CB74797880B41D79EA5B73B, taskId=task_1463524664473_0001_1_00_000000, startTime=1463524664669, finishTime=1463524664705, timeTaken=36, status=SUCCEEDED, successfulAttemptID=attempt_1463524664473_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524664698, Process_Duration=4, Process_End_Time=1463524664702, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:44,706 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524664473_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:44,706 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B] : 1
2016-05-17 22:37:44,706 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B]
2016-05-17 22:37:44,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664473_0001_1][Event:VERTEX_FINISHED]: vertexName=D6C2275D9CB74797880B41D79EA5B73B, vertexId=vertex_1463524664473_0001_1_00, initRequestedTime=1463524664642, initedTime=1463524664653, startRequestedTime=1463524664655, startedTime=1463524664655, finishTime=1463524664706, timeTaken=51, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524664698, Process_Duration=4, Process_End_Time=1463524664702, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524664669, firstTasksToStart=[ task_1463524664473_0001_1_00_000000 ], lastTaskFinishTime=1463524664704, lastTasksToFinish=[ task_1463524664473_0001_1_00_000000 ], minTaskDuration=35, maxTaskDuration=35, avgTaskDuration=35.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524664473_0001_1_00_000000 ], longestDurationTasks=[ task_1463524664473_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:44,707 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:44,708 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:44,708 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524664698, Process_Duration=4, Process_End_Time=1463524664702, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:44,708 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524664473_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:44,708 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:44,709 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:44,709 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664473_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524664473_0001_00_000001, stoppedTime=1463524664709, exitStatus=0
2016-05-17 22:37:44,709 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524664473_0001_00_000001
2016-05-17 22:37:44,709 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:44,709 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524664473_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:44,710 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524664473_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:44,710 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:44,710 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524664473_0001_1
2016-05-17 22:37:44,711 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664473_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524664473_0001_1
2016-05-17 22:37:44,711 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524664473_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:44,711 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524664473_0001_00_000001, Exiting
2016-05-17 22:37:44,711 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524664473_0001_00_000001
2016-05-17 22:37:44,711 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524664473_0001_00_000001
2016-05-17 22:37:44,711 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0B5C61320AAB40BCBD546552B3FF9A2B for vertex: vertex_1463524664473_0001_1_00 [D6C2275D9CB74797880B41D79EA5B73B]
2016-05-17 22:37:44,712 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524664473_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:44,712 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524664473_0001_00_000001 completed successfully
2016-05-17 22:37:44,719 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524664473_0001_1, queueSize=0
2016-05-17 22:37:44,719 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:44,719 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664473_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524664473_0001_1, startTime=1463524664641, finishTime=1463524664718, timeTaken=77, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524664698, Process_Duration=4, Process_End_Time=1463524664702, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:44,719 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524664473_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:44,720 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524664473_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:44,720 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524664473_0001_00_000001
2016-05-17 22:37:44,720 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524664473_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:44 Completed Dag: dag_1463524664473_0001_1
2016-05-17 22:37:44,721 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524664473_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:44,721 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524664473_0001_1_post
2016-05-17 22:37:44,721 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:44,721 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:44,727 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...eld'_RESULTS.0_UNKNOWN_-1, applicationId=application_1463524664473_0001
2016-05-17 22:37:44,729 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:44,729 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:44,729 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:44,729 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:44,729 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:44,729 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:44,730 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:44,731 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:44,731 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:44,731 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:44,732 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:44,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:44,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:44,735 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/BF535445E13A49299B0BA4E74B00B602/DF815BE1896B45CFBC7C8A38BFA6B942/.tez/application_1463524664473_0001
2016-05-17 22:37:44,736 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_-1/_temporary
2016-05-17 22:37:44,736 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.285
2016-05-17 22:37:44,800 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_-1
2016-05-17 22:37:44,821 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:44,822 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:44,847 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 22:37:44,849 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-17 22:37:44,850 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:44,850 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:44,851 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_'field2'
2016-05-17 22:37:44,871 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:44,872 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:44,879 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:44,879 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:44,908 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 22:37:44,914 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 22:37:44,915 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:44,915 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:44,916 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:44,929 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:44,929 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS
2016-05-17 22:37:44,937 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:44,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:44,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS"]
2016-05-17 22:37:44,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:44,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:44,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:44,939 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-17 22:37:44,939 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:44,951 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:44,956 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:44,956 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:44,957 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:44,966 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6FF15B76453445B1A8D663732D646286/F632CC29DC2B4C92BC5CBD937D030808/.tez/application_1463524664956_0001 doesn't exist and is created
2016-05-17 22:37:44,982 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:44,982 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:44,982 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6FF15B76453445B1A8D663732D646286/F632CC29DC2B4C92BC5CBD937D030808/.tez/application_1463524664956_0001
2016-05-17 22:37:44,985 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524664956_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:45,006 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:45,007 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:45,009 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:45,009 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:45,009 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6FF15B76453445B1A8D663732D646286/F632CC29DC2B4C92BC5CBD937D030808/.tez/application_1463524664956_0001/localmode-log-dir
2016-05-17 22:37:45,010 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6FF15B76453445B1A8D663732D646286/F632CC29DC2B4C92BC5CBD937D030808/.tez/application_1463524664956_0001/localmode-log-dir/history.txt.appattempt_1463524664956_0001_000000, maxErrors=10
2016-05-17 22:37:45,010 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:45,010 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524664956_0001_000000, appSubmitTime=1463524664985, launchTime=1463524664985
2016-05-17 22:37:45,016 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:45,016 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:45,017 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:45,020 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:46417
2016-05-17 22:37:45,020 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:45,021 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:45,024 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524664956_0001_000000, startTime=1463524665024
2016-05-17 22:37:45,024 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:45,024 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:45,082 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:45,082 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:45,083 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...RESULTS.0_UNKNOWN_RESULTS, applicationId=application_1463524664956_0001, dagName=[6FF15B76453445B1A8D663732D646286/F632CC29DC2B4C92BC5CBD937D030808] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-17 22:37:45,099 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [6FF15B76453445B1A8D663732D646286/F632CC29DC2B4C92BC5CBD937D030808] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-17 22:37:45,101 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524664956_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6FF15B76453445B1A8D663732D646286/F632CC29DC2B4C92BC5CBD937D030808/.tez/application_1463524664956_0001/localmode-log-dir/dag_1463524664956_0001_1.dot
2016-05-17 22:37:45,101 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6FF15B76453445B1A8D663732D646286/F632CC29DC2B4C92BC5CBD937D030808/.tez/application_1463524664956_0001/localmode-log-dir/dag_1463524664956_0001_1-tez-dag.pb.txt
2016-05-17 22:37:45,105 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524664956_0001_1
2016-05-17 22:37:45,105 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:45,105 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [6FF15B76453445B1A8D663732D646286/F632CC29DC2B4C92BC5CBD937D030808] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-17 22:37:45 Running Dag: dag_1463524664956_0001_1
2016-05-17 22:37:45,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664956_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524664956_0001_1, submitTime=1463524665100
2016-05-17 22:37:45,117 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:45,118 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E5DE70A887EC419ABE39F864FCC4CE3F
2016-05-17 22:37:45,118 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:45,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664956_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524664956_0001_1, initTime=1463524665117
2016-05-17 22:37:45,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:45,119 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524664956_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:45,119 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664956_0001_1][Event:DAG_STARTED]: dagID=dag_1463524664956_0001_1, startTime=1463524665119
2016-05-17 22:37:45,119 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:45,119 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:45,119 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524664956_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:45,119 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E5DE70A887EC419ABE39F864FCC4CE3F : {4771513F39BB47A89F333FD6C9283B77={InputName=4771513F39BB47A89F333FD6C9283B77}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:45,119 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4771513F39BB47A89F333FD6C9283B77, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:45,120 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F]
2016-05-17 22:37:45,120 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F]
2016-05-17 22:37:45,120 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F]
2016-05-17 22:37:45,120 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F]. Starting root input initializers: 1
2016-05-17 22:37:45,122 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:45,122 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4771513F39BB47A89F333FD6C9283B77 on vertex vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F]
2016-05-17 22:37:45,123 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...RESULTS.0_UNKNOWN_RESULTS, applicationId=application_1463524664956_0001, dagName=[6FF15B76453445B1A8D663732D646286/F632CC29DC2B4C92BC5CBD937D030808] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS
2016-05-17 22:37:45,126 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:45,126 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4771513F39BB47A89F333FD6C9283B77 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:45,127 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:45,127 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:45,129 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:45,129 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4771513F39BB47A89F333FD6C9283B77 on vertex vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F]
2016-05-17 22:37:45,129 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:45,130 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4771513F39BB47A89F333FD6C9283B77=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:45,130 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F] parallelism set to 1
2016-05-17 22:37:45,130 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:45,130 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664956_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524664956_0001_1_00, eventCount=1
2016-05-17 22:37:45,130 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F]
2016-05-17 22:37:45,131 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=11E2A0BE51974E318061BACC47D85C16, vertexId=vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:45,131 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=11E2A0BE51974E318061BACC47D85C16, vertexId=vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F]
2016-05-17 22:37:45,133 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=11E2A0BE51974E318061BACC47D85C16, vertexName=E5DE70A887EC419ABE39F864FCC4CE3F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:45,133 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=11E2A0BE51974E318061BACC47D85C16, vertexId=vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F]
2016-05-17 22:37:45,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664956_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E5DE70A887EC419ABE39F864FCC4CE3F, vertexId=vertex_1463524664956_0001_1_00, initRequestedTime=1463524665119, initedTime=1463524665134, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:45,134 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:45,134 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F] with distanceFromRoot: 0
2016-05-17 22:37:45,135 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:45,135 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E5DE70A887EC419ABE39F864FCC4CE3F
2016-05-17 22:37:45,135 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F]
2016-05-17 22:37:45,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664956_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524664956_0001_1_00, startRequestedTime=1463524665135, startedTime=1463524665135
2016-05-17 22:37:45,136 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:45,137 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:45,137 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:45,138 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664956_0001_1][Event:TASK_STARTED]: vertexName=E5DE70A887EC419ABE39F864FCC4CE3F, taskId=task_1463524664956_0001_1_00_000000, scheduledTime=1463524665137, launchTime=1463524665137
2016-05-17 22:37:45,138 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524664956_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:45,138 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F]
2016-05-17 22:37:45,138 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6FF15B76453445B1A8D663732D646286/F632CC29DC2B4C92BC5CBD937D030808] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS, VertexName: E5DE70A887EC419ABE39F864FCC4CE3F, VertexParallelism: 1, TaskAttemptID:attempt_1463524664956_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4771513F39BB47A89F333FD6C9283B77, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=11E2A0BE51974E318061BACC47D85C16, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:45,139 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524664956_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:45,139 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:45,140 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:45,140 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:45,140 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:45,141 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524664956_0001_00_000001
2016-05-17 22:37:45,141 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524664956_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:45,142 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524664956_0001, with dag id: dag_1463524664956_0001_1
2016-05-17 22:37:45,142 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524664956_0001_00_000001
2016-05-17 22:37:45,143 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:45,143 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524664956_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:45,144 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:45,145 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664956_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524664956_0001_00_000001, launchTime=1463524665143
2016-05-17 22:37:45,146 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524664956_0001_00_000001
2016-05-17 22:37:45,147 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524664956_0001_1_00_000000_0] to container: [container_1463524664956_0001_00_000001]
2016-05-17 22:37:45,147 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524664956_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:45,147 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524664956_0001_00_000001 given task: attempt_1463524664956_0001_1_00_000000_0
2016-05-17 22:37:45,147 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524664956_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524664956_0001_1_00_000000_0
2016-05-17 22:37:45 Starting to run new task attempt: attempt_1463524664956_0001_1_00_000000_0
2016-05-17 22:37:45,148 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524664956_0001_1_00_000000_0
2016-05-17 22:37:45,148 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:45,148 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:45,148 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524664956_0001_1_00_000000_0] started. Is using containerId: [container_1463524664956_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:45,150 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:45,150 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:45,150 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:45,150 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:45,150 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:45,151 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6FF15B76453445B1A8D663732D646286/F632CC29DC2B4C92BC5CBD937D030808] every.every/(1/1) ...RESULTS.0_UNKNOWN_RESULTS, VertexName: E5DE70A887EC419ABE39F864FCC4CE3F, VertexParallelism: 1, TaskAttemptID:attempt_1463524664956_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4771513F39BB47A89F333FD6C9283B77, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=11E2A0BE51974E318061BACC47D85C16, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:45,152 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:45,150 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664956_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E5DE70A887EC419ABE39F864FCC4CE3F, taskAttemptId=attempt_1463524664956_0001_1_00_000000_0, startTime=1463524665148, containerId=container_1463524664956_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524664956_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:45,152 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524664956_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:45,152 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524664956_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:45,153 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524664956_0001_1_00_000000_0
2016-05-17 22:37:45,154 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:45,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:45,155 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524664956_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:45,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4771513F39BB47A89F333FD6C9283B77, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:45,156 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=11E2A0BE51974E318061BACC47D85C16, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:45,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:45,156 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:45,156 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 11E2A0BE51974E318061BACC47D85C16
2016-05-17 22:37:45,156 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:45,157 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 11E2A0BE51974E318061BACC47D85C16
2016-05-17 22:37:45,158 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:45,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4771513F39BB47A89F333FD6C9283B77
2016-05-17 22:37:45,159 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4771513F39BB47A89F333FD6C9283B77
2016-05-17 22:37:45,159 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:45,165 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:45,165 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4771513F39BB47A89F333FD6C9283B77
2016-05-17 22:37:45,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4771513F39BB47A89F333FD6C9283B77
2016-05-17 22:37:45,169 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E5DE70A887EC419ABE39F864FCC4CE3F, ordinal: 0
2016-05-17 22:37:45,169 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E5DE70A887EC419ABE39F864FCC4CE3F, mem on start (mb), free: 318, total: 386, max: 672
2016-05-17 22:37:45,169 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:45,170 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:45,170 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:45,173 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:45,173 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 11E2A0BE51974E318061BACC47D85C16
2016-05-17 22:37:45,173 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:45,174 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:45,174 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:45,174 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:45,174 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:45,174 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:45,174 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:45,174 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:45,175 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 11E2A0BE51974E318061BACC47D85C16, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:45,175 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4771513F39BB47A89F333FD6C9283B77, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:45,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:45,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4771513F39BB47A89F333FD6C9283B77 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:45,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4771513F39BB47A89F333FD6C9283B77
2016-05-17 22:37:45,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:45,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:45,175 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:45,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4771513F39BB47A89F333FD6C9283B77
2016-05-17 22:37:45,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:45,176 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524664956_0001_1_00_000000_0
2016-05-17 22:37:45,176 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:45,177 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@13ec6cf
2016-05-17 22:37:45,177 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@13ec6cf
2016-05-17 22:37:45,178 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:45,178 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:45,179 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:45,179 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 4771513F39BB47A89F333FD6C9283B77
2016-05-17 22:37:45,180 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS"], id: 11E2A0BE51974E318061BACC47D85C16
2016-05-17 22:37:45,180 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS"]
2016-05-17 22:37:45,180 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:45,181 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:45,181 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E5DE70A887EC419ABE39F864FCC4CE3F, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:45,184 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:45,184 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:45,184 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524664956_0001_1_00_000000_0
2016-05-17 22:37:45,184 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524664956_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:45,185 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524664956_0001_1_00_000000_0
2016-05-17 22:37:45,185 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E5DE70A887EC419ABE39F864FCC4CE3F, mem on close (mb), free: 317, total: 386, max: 672
2016-05-17 22:37:45 Completed running task attempt: attempt_1463524664956_0001_1_00_000000_0
2016-05-17 22:37:45,188 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524664956_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:45,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524665181, Process_Duration=4, Process_End_Time=1463524665185, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:45,189 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664956_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E5DE70A887EC419ABE39F864FCC4CE3F, taskAttemptId=attempt_1463524664956_0001_1_00_000000_0, startTime=1463524665148, finishTime=1463524665189, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524665181, Process_Duration=4, Process_End_Time=1463524665185, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:45,189 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:45,190 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524664956_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:45,190 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:45,190 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524664956_0001_1_00_000000_0
2016-05-17 22:37:45,190 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:45,190 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:45,191 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:45,191 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524664956_0001_00_000001
2016-05-17 22:37:45,190 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664956_0001_1][Event:TASK_FINISHED]: vertexName=E5DE70A887EC419ABE39F864FCC4CE3F, taskId=task_1463524664956_0001_1_00_000000, startTime=1463524665148, finishTime=1463524665190, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463524664956_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524665181, Process_Duration=4, Process_End_Time=1463524665185, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:45,192 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524664956_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:45,192 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F] : 1
2016-05-17 22:37:45,192 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F]
2016-05-17 22:37:45,193 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:45,193 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664956_0001_1][Event:VERTEX_FINISHED]: vertexName=E5DE70A887EC419ABE39F864FCC4CE3F, vertexId=vertex_1463524664956_0001_1_00, initRequestedTime=1463524665119, initedTime=1463524665134, startRequestedTime=1463524665135, startedTime=1463524665135, finishTime=1463524665192, timeTaken=57, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524665181, Process_Duration=4, Process_End_Time=1463524665185, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524665148, firstTasksToStart=[ task_1463524664956_0001_1_00_000000 ], lastTaskFinishTime=1463524665189, lastTasksToFinish=[ task_1463524664956_0001_1_00_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524664956_0001_1_00_000000 ], longestDurationTasks=[ task_1463524664956_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:45,193 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:45,193 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524664956_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:45,194 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664956_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524664956_0001_00_000001, stoppedTime=1463524665194, exitStatus=0
2016-05-17 22:37:45,194 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:45,194 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524664956_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:45,194 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:45,194 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524664956_0001_1
2016-05-17 22:37:45,194 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664956_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524664956_0001_1
2016-05-17 22:37:45,195 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 11E2A0BE51974E318061BACC47D85C16 for vertex: vertex_1463524664956_0001_1_00 [E5DE70A887EC419ABE39F864FCC4CE3F]
2016-05-17 22:37:45,202 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524664956_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:45,202 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524664956_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:45,202 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524664956_0001_00_000001, Exiting
2016-05-17 22:37:45,202 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524664956_0001_00_000001
2016-05-17 22:37:45,202 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524664956_0001_00_000001
2016-05-17 22:37:45,203 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524664956_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:45,203 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524664956_0001_00_000001 completed successfully
2016-05-17 22:37:45,204 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524664956_0001_1, queueSize=0
2016-05-17 22:37:45,204 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:45,205 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524664956_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524664956_0001_1, startTime=1463524665119, finishTime=1463524665202, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524665181, Process_Duration=4, Process_End_Time=1463524665185, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:45,205 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524664956_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:45,205 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524664956_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:45,205 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524664956_0001_00_000001
2016-05-17 22:37:45,205 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524664956_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:45 Completed Dag: dag_1463524664956_0001_1
2016-05-17 22:37:45,206 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524664956_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:45,207 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524664956_0001_1_post
2016-05-17 22:37:45,207 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:45,208 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:45,211 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...RESULTS.0_UNKNOWN_RESULTS, applicationId=application_1463524664956_0001
2016-05-17 22:37:45,213 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:45,213 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:45,213 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:45,213 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:45,213 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:45,214 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:45,214 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:45,214 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:45,215 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:45,216 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:45,216 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:45,216 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:45,216 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:45,221 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6FF15B76453445B1A8D663732D646286/F632CC29DC2B4C92BC5CBD937D030808/.tez/application_1463524664956_0001
2016-05-17 22:37:45,222 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_RESULTS/_temporary
2016-05-17 22:37:45,223 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.285
2016-05-17 22:37:45,289 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_RESULTS
2016-05-17 22:37:45,310 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:45,311 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:45,342 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 22:37:45,347 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 22:37:45,347 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:45,348 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:45,348 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:45,361 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:45,362 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL
2016-05-17 22:37:45,371 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:45,371 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:45,372 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL"]
2016-05-17 22:37:45,372 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:45,372 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:45,372 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:45,374 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-17 22:37:45,374 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:45,387 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:45,387 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:45,391 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:45,396 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:45,397 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:45,397 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:45,407 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7FBCEB2A529A40EDA9397FB2C7396431/B35CAA4D1ED04C809A23387C8A4355B0/.tez/application_1463524665396_0001 doesn't exist and is created
2016-05-17 22:37:45,424 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:45,424 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:45,425 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7FBCEB2A529A40EDA9397FB2C7396431/B35CAA4D1ED04C809A23387C8A4355B0/.tez/application_1463524665396_0001
2016-05-17 22:37:45,428 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524665396_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:45,441 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:45,442 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:45,444 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:45,444 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:45,444 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7FBCEB2A529A40EDA9397FB2C7396431/B35CAA4D1ED04C809A23387C8A4355B0/.tez/application_1463524665396_0001/localmode-log-dir
2016-05-17 22:37:45,444 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7FBCEB2A529A40EDA9397FB2C7396431/B35CAA4D1ED04C809A23387C8A4355B0/.tez/application_1463524665396_0001/localmode-log-dir/history.txt.appattempt_1463524665396_0001_000000, maxErrors=10
2016-05-17 22:37:45,444 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:45,445 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524665396_0001_000000, appSubmitTime=1463524665428, launchTime=1463524665428
2016-05-17 22:37:45,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:45,451 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:45,452 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:45,452 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:45,458 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:39883
2016-05-17 22:37:45,460 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:45,462 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524665396_0001_000000, startTime=1463524665462
2016-05-17 22:37:45,462 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:45,462 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:45,525 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:45,525 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:45,525 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, applicationId=application_1463524665396_0001, dagName=[7FBCEB2A529A40EDA9397FB2C7396431/B35CAA4D1ED04C809A23387C8A4355B0] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-17 22:37:45,540 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7FBCEB2A529A40EDA9397FB2C7396431/B35CAA4D1ED04C809A23387C8A4355B0] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-17 22:37:45,541 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524665396_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7FBCEB2A529A40EDA9397FB2C7396431/B35CAA4D1ED04C809A23387C8A4355B0/.tez/application_1463524665396_0001/localmode-log-dir/dag_1463524665396_0001_1.dot
2016-05-17 22:37:45,541 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7FBCEB2A529A40EDA9397FB2C7396431/B35CAA4D1ED04C809A23387C8A4355B0/.tez/application_1463524665396_0001/localmode-log-dir/dag_1463524665396_0001_1-tez-dag.pb.txt
2016-05-17 22:37:45,545 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524665396_0001_1
2016-05-17 22:37:45,546 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:45,546 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7FBCEB2A529A40EDA9397FB2C7396431/B35CAA4D1ED04C809A23387C8A4355B0] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-17 22:37:45 Running Dag: dag_1463524665396_0001_1
2016-05-17 22:37:45,557 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665396_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524665396_0001_1, submitTime=1463524665540
2016-05-17 22:37:45,557 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:45,557 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BC3DA42271174E1FBDAF20DA0FFB879C
2016-05-17 22:37:45,558 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:45,559 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:45,559 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665396_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524665396_0001_1, initTime=1463524665557
2016-05-17 22:37:45,559 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524665396_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:45,560 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665396_0001_1][Event:DAG_STARTED]: dagID=dag_1463524665396_0001_1, startTime=1463524665559
2016-05-17 22:37:45,560 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:45,560 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524665396_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:45,560 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:45,560 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BC3DA42271174E1FBDAF20DA0FFB879C : {EE71A1FB02914A5793535E2EC410D569={InputName=EE71A1FB02914A5793535E2EC410D569}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:45,560 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: EE71A1FB02914A5793535E2EC410D569, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:45,560 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C]
2016-05-17 22:37:45,561 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C]
2016-05-17 22:37:45,561 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C]
2016-05-17 22:37:45,561 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C]. Starting root input initializers: 1
2016-05-17 22:37:45,561 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:45,562 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: EE71A1FB02914A5793535E2EC410D569 on vertex vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C]
2016-05-17 22:37:45,562 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, applicationId=application_1463524665396_0001, dagName=[7FBCEB2A529A40EDA9397FB2C7396431/B35CAA4D1ED04C809A23387C8A4355B0] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL
2016-05-17 22:37:45,563 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:45,563 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input EE71A1FB02914A5793535E2EC410D569 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:45,563 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:45,563 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:45,564 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:45,565 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: EE71A1FB02914A5793535E2EC410D569 on vertex vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C]
2016-05-17 22:37:45,565 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:45,565 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {EE71A1FB02914A5793535E2EC410D569=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:45,565 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C] parallelism set to 1
2016-05-17 22:37:45,565 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:45,566 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665396_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524665396_0001_1_00, eventCount=1
2016-05-17 22:37:45,566 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C]
2016-05-17 22:37:45,566 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9D7B1981FD3F422CA44B0AD1C7C5FA49, vertexId=vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:45,566 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9D7B1981FD3F422CA44B0AD1C7C5FA49, vertexId=vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C]
2016-05-17 22:37:45,567 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9D7B1981FD3F422CA44B0AD1C7C5FA49, vertexName=BC3DA42271174E1FBDAF20DA0FFB879C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:45,568 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9D7B1981FD3F422CA44B0AD1C7C5FA49, vertexId=vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C]
2016-05-17 22:37:45,569 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665396_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BC3DA42271174E1FBDAF20DA0FFB879C, vertexId=vertex_1463524665396_0001_1_00, initRequestedTime=1463524665560, initedTime=1463524665569, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:45,569 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C] with distanceFromRoot: 0
2016-05-17 22:37:45,569 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:45,569 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BC3DA42271174E1FBDAF20DA0FFB879C
2016-05-17 22:37:45,569 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:45,569 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C]
2016-05-17 22:37:45,570 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:45,570 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665396_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524665396_0001_1_00, startRequestedTime=1463524665569, startedTime=1463524665569
2016-05-17 22:37:45,571 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:45,572 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:45,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665396_0001_1][Event:TASK_STARTED]: vertexName=BC3DA42271174E1FBDAF20DA0FFB879C, taskId=task_1463524665396_0001_1_00_000000, scheduledTime=1463524665571, launchTime=1463524665571
2016-05-17 22:37:45,572 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524665396_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:45,572 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C]
2016-05-17 22:37:45,573 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7FBCEB2A529A40EDA9397FB2C7396431/B35CAA4D1ED04C809A23387C8A4355B0] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, VertexName: BC3DA42271174E1FBDAF20DA0FFB879C, VertexParallelism: 1, TaskAttemptID:attempt_1463524665396_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EE71A1FB02914A5793535E2EC410D569, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9D7B1981FD3F422CA44B0AD1C7C5FA49, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:45,573 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524665396_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:45,573 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:45,574 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:45,574 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:45,574 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:45,575 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524665396_0001_00_000001
2016-05-17 22:37:45,575 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524665396_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:45,576 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524665396_0001_00_000001
2016-05-17 22:37:45,576 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:45,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:45,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665396_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524665396_0001_00_000001, launchTime=1463524665577
2016-05-17 22:37:45,579 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524665396_0001_00_000001
2016-05-17 22:37:45,579 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524665396_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:45,580 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524665396_0001_1_00_000000_0] to container: [container_1463524665396_0001_00_000001]
2016-05-17 22:37:45,580 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524665396_0001, with dag id: dag_1463524665396_0001_1
2016-05-17 22:37:45,581 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524665396_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:45,581 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524665396_0001_00_000001 given task: attempt_1463524665396_0001_1_00_000000_0
2016-05-17 22:37:45,581 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524665396_0001_1_00_000000_0] started. Is using containerId: [container_1463524665396_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:45,582 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524665396_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524665396_0001_1_00_000000_0
2016-05-17 22:37:45,582 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:45 Starting to run new task attempt: attempt_1463524665396_0001_1_00_000000_0
2016-05-17 22:37:45,582 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524665396_0001_1_00_000000_0
2016-05-17 22:37:45,582 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665396_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BC3DA42271174E1FBDAF20DA0FFB879C, taskAttemptId=attempt_1463524665396_0001_1_00_000000_0, startTime=1463524665581, containerId=container_1463524665396_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524665396_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:45,583 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524665396_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:45,582 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:45,584 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:45,584 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:45,584 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:45,584 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524665396_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:45,584 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:45,584 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:45,584 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7FBCEB2A529A40EDA9397FB2C7396431/B35CAA4D1ED04C809A23387C8A4355B0] every.every/(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, VertexName: BC3DA42271174E1FBDAF20DA0FFB879C, VertexParallelism: 1, TaskAttemptID:attempt_1463524665396_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EE71A1FB02914A5793535E2EC410D569, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9D7B1981FD3F422CA44B0AD1C7C5FA49, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:45,585 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:45,585 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524665396_0001_1_00_000000_0
2016-05-17 22:37:45,586 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:45,586 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:45,587 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EE71A1FB02914A5793535E2EC410D569, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:45,588 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524665396_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:45,587 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:45,587 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9D7B1981FD3F422CA44B0AD1C7C5FA49, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:45,588 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:45,589 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EE71A1FB02914A5793535E2EC410D569
2016-05-17 22:37:45,589 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:45,589 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: EE71A1FB02914A5793535E2EC410D569
2016-05-17 22:37:45,590 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9D7B1981FD3F422CA44B0AD1C7C5FA49
2016-05-17 22:37:45,590 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:45,590 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9D7B1981FD3F422CA44B0AD1C7C5FA49
2016-05-17 22:37:45,592 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:45,592 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:45,593 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: EE71A1FB02914A5793535E2EC410D569
2016-05-17 22:37:45,593 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EE71A1FB02914A5793535E2EC410D569
2016-05-17 22:37:45,599 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BC3DA42271174E1FBDAF20DA0FFB879C, ordinal: 0
2016-05-17 22:37:45,599 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BC3DA42271174E1FBDAF20DA0FFB879C, mem on start (mb), free: 239, total: 386, max: 672
2016-05-17 22:37:45,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:45,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:45,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:45,602 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:45,602 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9D7B1981FD3F422CA44B0AD1C7C5FA49
2016-05-17 22:37:45,602 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:45,602 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:45,602 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:45,602 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:45,603 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:45,603 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:45,603 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:45,603 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:45,604 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EE71A1FB02914A5793535E2EC410D569, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:45,604 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9D7B1981FD3F422CA44B0AD1C7C5FA49, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:45,604 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:45,604 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EE71A1FB02914A5793535E2EC410D569 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:45,604 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EE71A1FB02914A5793535E2EC410D569
2016-05-17 22:37:45,604 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:45,604 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EE71A1FB02914A5793535E2EC410D569
2016-05-17 22:37:45,604 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:45,605 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:45,605 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:45,605 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524665396_0001_1_00_000000_0
2016-05-17 22:37:45,605 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:45,606 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@54355171
2016-05-17 22:37:45,607 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@54355171
2016-05-17 22:37:45,607 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:45,608 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:45,608 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:45,609 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: EE71A1FB02914A5793535E2EC410D569
2016-05-17 22:37:45,609 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL"], id: 9D7B1981FD3F422CA44B0AD1C7C5FA49
2016-05-17 22:37:45,609 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL"]
2016-05-17 22:37:45,609 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:45,610 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:45,610 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BC3DA42271174E1FBDAF20DA0FFB879C, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:45,613 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:45,613 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:45,613 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524665396_0001_1_00_000000_0
2016-05-17 22:37:45,614 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524665396_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:45,614 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524665396_0001_1_00_000000_0
2016-05-17 22:37:45,614 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BC3DA42271174E1FBDAF20DA0FFB879C, mem on close (mb), free: 238, total: 386, max: 672
2016-05-17 22:37:45 Completed running task attempt: attempt_1463524665396_0001_1_00_000000_0
2016-05-17 22:37:45,615 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524665396_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:45,617 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524665610, Process_Duration=4, Process_End_Time=1463524665614, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:45,617 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:45,617 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665396_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BC3DA42271174E1FBDAF20DA0FFB879C, taskAttemptId=attempt_1463524665396_0001_1_00_000000_0, startTime=1463524665581, finishTime=1463524665617, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524665610, Process_Duration=4, Process_End_Time=1463524665614, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:45,617 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:45,617 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524665396_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:45,617 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:45,618 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524665396_0001_1_00_000000_0
2016-05-17 22:37:45,618 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524665396_0001_00_000001
2016-05-17 22:37:45,618 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665396_0001_1][Event:TASK_FINISHED]: vertexName=BC3DA42271174E1FBDAF20DA0FFB879C, taskId=task_1463524665396_0001_1_00_000000, startTime=1463524665581, finishTime=1463524665618, timeTaken=37, status=SUCCEEDED, successfulAttemptID=attempt_1463524665396_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524665610, Process_Duration=4, Process_End_Time=1463524665614, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:45,618 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524665396_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:45,618 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:45,618 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C] : 1
2016-05-17 22:37:45,619 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C]
2016-05-17 22:37:45,619 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:45,619 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665396_0001_1][Event:VERTEX_FINISHED]: vertexName=BC3DA42271174E1FBDAF20DA0FFB879C, vertexId=vertex_1463524665396_0001_1_00, initRequestedTime=1463524665560, initedTime=1463524665569, startRequestedTime=1463524665569, startedTime=1463524665569, finishTime=1463524665619, timeTaken=50, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524665610, Process_Duration=4, Process_End_Time=1463524665614, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524665581, firstTasksToStart=[ task_1463524665396_0001_1_00_000000 ], lastTaskFinishTime=1463524665617, lastTasksToFinish=[ task_1463524665396_0001_1_00_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524665396_0001_1_00_000000 ], longestDurationTasks=[ task_1463524665396_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:45,620 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:45,620 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524665396_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:45,620 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:45,620 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524665396_0001_1
2016-05-17 22:37:45,620 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665396_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524665396_0001_1
2016-05-17 22:37:45,620 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 9D7B1981FD3F422CA44B0AD1C7C5FA49 for vertex: vertex_1463524665396_0001_1_00 [BC3DA42271174E1FBDAF20DA0FFB879C]
2016-05-17 22:37:45,619 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:45,630 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524665396_0001_1, queueSize=0
2016-05-17 22:37:45,630 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:45,630 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665396_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524665396_0001_1, startTime=1463524665559, finishTime=1463524665628, timeTaken=69, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=272, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524665610, Process_Duration=4, Process_End_Time=1463524665614, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:45,630 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524665396_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:45,630 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524665396_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:45,631 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665396_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524665396_0001_00_000001, stoppedTime=1463524665630, exitStatus=0
2016-05-17 22:37:45,631 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:45,631 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524665396_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:45 Completed Dag: dag_1463524665396_0001_1
2016-05-17 22:37:45,632 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524665396_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:45,632 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524665396_0001_1_post
2016-05-17 22:37:45,632 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:45,632 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:45,633 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524665396_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:37:45,634 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524665396_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:37:45,638 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ld'_RESULTS.0_UNKNOWN_ALL, applicationId=application_1463524665396_0001
2016-05-17 22:37:45,640 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:45,640 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:45,640 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:45,640 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:45,640 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:45,640 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:45,641 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:45,641 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:45,641 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:45,641 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:45,641 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:45,641 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:45,641 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:45,642 INFO task.TezChild (TezChild.java:run(198)) - Interrupted while waiting for new work for container container_1463524665396_0001_00_000001
2016-05-17 22:37:45,642 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524665396_0001_00_000001
2016-05-17 22:37:45,642 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524665396_0001_00_000001
2016-05-17 22:37:45,644 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524665396_0001_00_000001 = ContainerExecutionResult{exitStatus=INTERRUPTED, throwable=java.lang.InterruptedException, errorMessage='Interrupted while waiting for new work'}
2016-05-17 22:37:45,644 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524665396_0001_00_000001 completed but with errors
2016-05-17 22:37:45,645 WARN event.AsyncDispatcher (AsyncDispatcher.java:handle(243)) - AsyncDispatcher thread interrupted
java.lang.InterruptedException
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219)
at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340)
at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338)
at org.apache.hadoop.yarn.event.AsyncDispatcher$GenericEventHandler.handle(AsyncDispatcher.java:240)
at org.apache.tez.dag.app.launcher.LocalContainerLauncher$RunningTaskCallback.onSuccess(LocalContainerLauncher.java:280)
at org.apache.tez.dag.app.launcher.LocalContainerLauncher$RunningTaskCallback.onSuccess(LocalContainerLauncher.java:256)
at com.google.common.util.concurrent.Futures$6.run(Futures.java:975)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:37:45,648 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onFailure(294)) - Container: container_1463524665396_0001_00_000001: Execution Failed:
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.InterruptedException
at org.apache.hadoop.yarn.event.AsyncDispatcher$GenericEventHandler.handle(AsyncDispatcher.java:245)
at org.apache.tez.dag.app.launcher.LocalContainerLauncher$RunningTaskCallback.onSuccess(LocalContainerLauncher.java:280)
at org.apache.tez.dag.app.launcher.LocalContainerLauncher$RunningTaskCallback.onSuccess(LocalContainerLauncher.java:256)
at com.google.common.util.concurrent.Futures$6.run(Futures.java:975)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.InterruptedException
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219)
at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340)
at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338)
at org.apache.hadoop.yarn.event.AsyncDispatcher$GenericEventHandler.handle(AsyncDispatcher.java:240)
... 6 more
2016-05-17 22:37:45,648 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524665396_0001_00_000001 exited with diagnostics set to java.lang.InterruptedException
2016-05-17 22:37:45,648 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524665396_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:37:45,652 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/7FBCEB2A529A40EDA9397FB2C7396431/B35CAA4D1ED04C809A23387C8A4355B0/.tez/application_1463524665396_0001
2016-05-17 22:37:45,652 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_UNKNOWN_ALL/_temporary
2016-05-17 22:37:45,653 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.281
2016-05-17 22:37:45,725 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_UNKNOWN_ALL
2016-05-17 22:37:45,755 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:45,755 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:45,785 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 22:37:45,791 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 22:37:45,791 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:45,791 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:45,792 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:45,805 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:45,806 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1
2016-05-17 22:37:45,814 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:45,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:45,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1"]
2016-05-17 22:37:45,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:45,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:45,816 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:45,817 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-17 22:37:45,817 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:45,828 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:45,833 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:45,834 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:45,834 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:45,843 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5A6C8B36B6746A4A9A3D483FB674C4F/E12572A8897E456598FAB90BC985B2A7/.tez/application_1463524665833_0001 doesn't exist and is created
2016-05-17 22:37:45,858 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:45,859 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:45,859 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5A6C8B36B6746A4A9A3D483FB674C4F/E12572A8897E456598FAB90BC985B2A7/.tez/application_1463524665833_0001
2016-05-17 22:37:45,861 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524665833_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:45,871 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:45,871 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:45,873 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:45,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:45,873 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5A6C8B36B6746A4A9A3D483FB674C4F/E12572A8897E456598FAB90BC985B2A7/.tez/application_1463524665833_0001/localmode-log-dir
2016-05-17 22:37:45,873 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5A6C8B36B6746A4A9A3D483FB674C4F/E12572A8897E456598FAB90BC985B2A7/.tez/application_1463524665833_0001/localmode-log-dir/history.txt.appattempt_1463524665833_0001_000000, maxErrors=10
2016-05-17 22:37:45,873 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:45,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524665833_0001_000000, appSubmitTime=1463524665861, launchTime=1463524665861
2016-05-17 22:37:45,877 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:45,879 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:45,880 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:45,880 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:45,881 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:45,882 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:45,885 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:44939
2016-05-17 22:37:45,886 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:45,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524665833_0001_000000, startTime=1463524665888
2016-05-17 22:37:45,888 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:45,888 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:45,959 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:45,959 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:45,959 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ield'_RESULTS.0_VALUES_-1, applicationId=application_1463524665833_0001, dagName=[D5A6C8B36B6746A4A9A3D483FB674C4F/E12572A8897E456598FAB90BC985B2A7] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-17 22:37:45,974 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D5A6C8B36B6746A4A9A3D483FB674C4F/E12572A8897E456598FAB90BC985B2A7] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-17 22:37:45,975 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524665833_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5A6C8B36B6746A4A9A3D483FB674C4F/E12572A8897E456598FAB90BC985B2A7/.tez/application_1463524665833_0001/localmode-log-dir/dag_1463524665833_0001_1.dot
2016-05-17 22:37:45,975 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5A6C8B36B6746A4A9A3D483FB674C4F/E12572A8897E456598FAB90BC985B2A7/.tez/application_1463524665833_0001/localmode-log-dir/dag_1463524665833_0001_1-tez-dag.pb.txt
2016-05-17 22:37:45,980 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524665833_0001_1
2016-05-17 22:37:45,980 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:45,980 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D5A6C8B36B6746A4A9A3D483FB674C4F/E12572A8897E456598FAB90BC985B2A7] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-17 22:37:45 Running Dag: dag_1463524665833_0001_1
2016-05-17 22:37:45,991 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665833_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524665833_0001_1, submitTime=1463524665974
2016-05-17 22:37:45,991 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:45,992 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F720084A13224C76825A400910AB2CF1
2016-05-17 22:37:45,992 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:45,993 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665833_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524665833_0001_1, initTime=1463524665992
2016-05-17 22:37:45,993 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524665833_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:45,995 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:45,995 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665833_0001_1][Event:DAG_STARTED]: dagID=dag_1463524665833_0001_1, startTime=1463524665993
2016-05-17 22:37:45,995 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:45,995 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524665833_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:45,995 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F720084A13224C76825A400910AB2CF1 : {F23F040E9B7A4E59ADC0825EF426DFAB={InputName=F23F040E9B7A4E59ADC0825EF426DFAB}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:45,996 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F23F040E9B7A4E59ADC0825EF426DFAB, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:45,996 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1]
2016-05-17 22:37:45,996 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1]
2016-05-17 22:37:45,996 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1]
2016-05-17 22:37:45,996 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1]. Starting root input initializers: 1
2016-05-17 22:37:45,997 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:45,997 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:45,997 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ield'_RESULTS.0_VALUES_-1, applicationId=application_1463524665833_0001, dagName=[D5A6C8B36B6746A4A9A3D483FB674C4F/E12572A8897E456598FAB90BC985B2A7] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-17 22:37:45,997 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F23F040E9B7A4E59ADC0825EF426DFAB on vertex vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1]
2016-05-17 22:37:45,999 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:45,999 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F23F040E9B7A4E59ADC0825EF426DFAB asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:45,999 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:46,000 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:46,001 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:46,001 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F23F040E9B7A4E59ADC0825EF426DFAB on vertex vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1]
2016-05-17 22:37:46,001 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:46,001 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F23F040E9B7A4E59ADC0825EF426DFAB=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:46,001 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1] parallelism set to 1
2016-05-17 22:37:46,001 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:46,002 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665833_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524665833_0001_1_00, eventCount=1
2016-05-17 22:37:46,002 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1]
2016-05-17 22:37:46,002 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=81AD2DD41E404726B52E0C801D272C55, vertexId=vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:46,002 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=81AD2DD41E404726B52E0C801D272C55, vertexId=vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1]
2016-05-17 22:37:46,004 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=81AD2DD41E404726B52E0C801D272C55, vertexName=F720084A13224C76825A400910AB2CF1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:46,004 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=81AD2DD41E404726B52E0C801D272C55, vertexId=vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1]
2016-05-17 22:37:46,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665833_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F720084A13224C76825A400910AB2CF1, vertexId=vertex_1463524665833_0001_1_00, initRequestedTime=1463524665995, initedTime=1463524666005, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:46,005 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1] with distanceFromRoot: 0
2016-05-17 22:37:46,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:46,005 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:46,006 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F720084A13224C76825A400910AB2CF1
2016-05-17 22:37:46,006 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1]
2016-05-17 22:37:46,006 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665833_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524665833_0001_1_00, startRequestedTime=1463524666006, startedTime=1463524666006
2016-05-17 22:37:46,006 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:46,006 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:46,007 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:46,007 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665833_0001_1][Event:TASK_STARTED]: vertexName=F720084A13224C76825A400910AB2CF1, taskId=task_1463524665833_0001_1_00_000000, scheduledTime=1463524666006, launchTime=1463524666006
2016-05-17 22:37:46,007 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524665833_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:46,007 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1]
2016-05-17 22:37:46,008 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D5A6C8B36B6746A4A9A3D483FB674C4F/E12572A8897E456598FAB90BC985B2A7] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1, VertexName: F720084A13224C76825A400910AB2CF1, VertexParallelism: 1, TaskAttemptID:attempt_1463524665833_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F23F040E9B7A4E59ADC0825EF426DFAB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=81AD2DD41E404726B52E0C801D272C55, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:46,008 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524665833_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:46,008 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:46,009 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:46,010 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:46,010 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:46,010 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524665833_0001_00_000001
2016-05-17 22:37:46,010 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524665833_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:46,012 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524665833_0001_00_000001
2016-05-17 22:37:46,012 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:46,012 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524665833_0001, with dag id: dag_1463524665833_0001_1
2016-05-17 22:37:46,014 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524665833_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:46,014 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:46,014 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665833_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524665833_0001_00_000001, launchTime=1463524666014
2016-05-17 22:37:46,016 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524665833_0001_00_000001
2016-05-17 22:37:46,017 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524665833_0001_1_00_000000_0] to container: [container_1463524665833_0001_00_000001]
2016-05-17 22:37:46,017 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524665833_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:46,017 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524665833_0001_00_000001 given task: attempt_1463524665833_0001_1_00_000000_0
2016-05-17 22:37:46,017 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524665833_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524665833_0001_1_00_000000_0
2016-05-17 22:37:46,017 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524665833_0001_1_00_000000_0] started. Is using containerId: [container_1463524665833_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:46,018 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:46,018 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665833_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F720084A13224C76825A400910AB2CF1, taskAttemptId=attempt_1463524665833_0001_1_00_000000_0, startTime=1463524666017, containerId=container_1463524665833_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524665833_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:46,018 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524665833_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:46,018 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524665833_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:46 Starting to run new task attempt: attempt_1463524665833_0001_1_00_000000_0
2016-05-17 22:37:46,019 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524665833_0001_1_00_000000_0
2016-05-17 22:37:46,019 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:46,019 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:46,019 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:46,020 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:46,020 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:46,020 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:46,020 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D5A6C8B36B6746A4A9A3D483FB674C4F/E12572A8897E456598FAB90BC985B2A7] every.every/(1/1) ...ield'_RESULTS.0_VALUES_-1, VertexName: F720084A13224C76825A400910AB2CF1, VertexParallelism: 1, TaskAttemptID:attempt_1463524665833_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F23F040E9B7A4E59ADC0825EF426DFAB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=81AD2DD41E404726B52E0C801D272C55, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:46,021 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:46,021 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524665833_0001_1_00_000000_0
2016-05-17 22:37:46,021 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:46,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:46,022 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524665833_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:46,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F23F040E9B7A4E59ADC0825EF426DFAB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:46,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=81AD2DD41E404726B52E0C801D272C55, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:46,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:46,024 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:46,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:46,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 81AD2DD41E404726B52E0C801D272C55
2016-05-17 22:37:46,025 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:46,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F23F040E9B7A4E59ADC0825EF426DFAB
2016-05-17 22:37:46,025 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F23F040E9B7A4E59ADC0825EF426DFAB
2016-05-17 22:37:46,025 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 81AD2DD41E404726B52E0C801D272C55
2016-05-17 22:37:46,028 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:46,028 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F23F040E9B7A4E59ADC0825EF426DFAB
2016-05-17 22:37:46,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F23F040E9B7A4E59ADC0825EF426DFAB
2016-05-17 22:37:46,028 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:46,034 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F720084A13224C76825A400910AB2CF1, ordinal: 0
2016-05-17 22:37:46,034 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F720084A13224C76825A400910AB2CF1, mem on start (mb), free: 163, total: 386, max: 672
2016-05-17 22:37:46,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:46,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:46,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:46,039 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:46,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 81AD2DD41E404726B52E0C801D272C55
2016-05-17 22:37:46,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:46,039 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:46,040 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:46,040 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:46,040 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:46,040 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:46,040 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:46,040 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:46,040 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F23F040E9B7A4E59ADC0825EF426DFAB, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:46,040 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 81AD2DD41E404726B52E0C801D272C55, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:46,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:46,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F23F040E9B7A4E59ADC0825EF426DFAB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:46,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F23F040E9B7A4E59ADC0825EF426DFAB
2016-05-17 22:37:46,041 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:46,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F23F040E9B7A4E59ADC0825EF426DFAB
2016-05-17 22:37:46,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:46,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:46,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:46,042 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524665833_0001_1_00_000000_0
2016-05-17 22:37:46,043 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:46,044 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@691cc451
2016-05-17 22:37:46,045 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:46,045 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F23F040E9B7A4E59ADC0825EF426DFAB
2016-05-17 22:37:46,045 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1"], id: 81AD2DD41E404726B52E0C801D272C55
2016-05-17 22:37:46,046 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1"]
2016-05-17 22:37:46,046 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:46,046 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:46,045 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@691cc451
2016-05-17 22:37:46,047 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:46,047 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:46,047 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F720084A13224C76825A400910AB2CF1, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:46,048 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:46,052 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:46,057 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:46,057 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:46,057 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:46,058 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524665833_0001_1_00_000000_0
2016-05-17 22:37:46,058 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524665833_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:46,058 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524665833_0001_1_00_000000_0
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:46,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524666048, Process_Duration=10, Process_End_Time=1463524666058, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:37:46,065 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F720084A13224C76825A400910AB2CF1, mem on close (mb), free: 161, total: 386, max: 672
2016-05-17 22:37:46,065 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:46,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665833_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F720084A13224C76825A400910AB2CF1, taskAttemptId=attempt_1463524665833_0001_1_00_000000_0, startTime=1463524666017, finishTime=1463524666065, timeTaken=48, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=404750336, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524666048, Process_Duration=10, Process_End_Time=1463524666058, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:37:46,066 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524665833_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:37:46,066 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524665833_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:37:46,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=F720084A13224C76825A400910AB2CF1, index=0
2016-05-17 22:37:46,066 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665833_0001_1][Event:TASK_FINISHED]: vertexName=F720084A13224C76825A400910AB2CF1, taskId=task_1463524665833_0001_1_00_000000, startTime=1463524666017, finishTime=1463524666066, timeTaken=49, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
], counters=Counters: 0
2016-05-17 22:37:46,067 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524665833_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:37:46,067 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1] : 1
2016-05-17 22:37:46,067 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1] because task failed: task_1463524665833_0001_1_00_000000
2016-05-17 22:37:46,067 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:37:46,067 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:37:46,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=F720084A13224C76825A400910AB2CF1, sourceVertex=F23F040E9B7A4E59ADC0825EF426DFAB
2016-05-17 22:37:46,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:46,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=F720084A13224C76825A400910AB2CF1, sourceVertex=81AD2DD41E404726B52E0C801D272C55
2016-05-17 22:37:46,068 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524665833_0001_00_000001
2016-05-17 22:37:46,069 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524665833_0001_00_000001
2016-05-17 22:37:46,068 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1]
2016-05-17 22:37:46,069 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=81AD2DD41E404726B52E0C801D272C55, vertexId=vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1]
2016-05-17 22:37:46,067 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:46,070 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:46,070 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665833_0001_1][Event:VERTEX_FINISHED]: vertexName=F720084A13224C76825A400910AB2CF1, vertexId=vertex_1463524665833_0001_1_00, initRequestedTime=1463524665995, initedTime=1463524666005, startRequestedTime=1463524666006, startedTime=1463524666006, finishTime=1463524666070, timeTaken=64, status=FAILED, diagnostics=Task failed, taskId=task_1463524665833_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524666017, firstTasksToStart=[ task_1463524665833_0001_1_00_000000 ], lastTaskFinishTime=1463524666065, lastTasksToFinish=[ task_1463524665833_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:37:46,071 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:37:46,071 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:37:46,071 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:37:46,071 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524665833_0001_1
2016-05-17 22:37:46,071 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 81AD2DD41E404726B52E0C801D272C55 for vertex: vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1]
2016-05-17 22:37:46,072 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524665833_0001_1, queueSize=0
2016-05-17 22:37:46,072 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665833_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524665833_0001_1, startTime=1463524665993, finishTime=1463524666072, timeTaken=79, status=FAILED, diagnostics=Vertex failed, vertexName=F720084A13224C76825A400910AB2CF1, vertexId=vertex_1463524665833_0001_1_00, diagnostics=[Task failed, taskId=task_1463524665833_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524665833_0001_1_00 [F720084A13224C76825A400910AB2CF1] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:37:46,073 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:46,073 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524665833_0001_1 finished with state: FAILED
2016-05-17 22:37:46,074 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524665833_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:46,083 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524665833_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:37:46,083 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524665833_0001_00_000001 completed but with errors
2016-05-17 22:37:46 Completed Dag: dag_1463524665833_0001_1
2016-05-17 22:37:46,090 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524665833_0001_1, dagState=FAILED
2016-05-17 22:37:46,090 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524665833_0001_1_post
2016-05-17 22:37:46,090 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:46,090 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:46,090 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524665833_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524665833_0001_00_000001, stoppedTime=1463524666090, exitStatus=0
2016-05-17 22:37:46,091 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524665833_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:46,091 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524665833_0001_1_00_000000_0 failed: true container: container_1463524665833_0001_00_000001 numFailedTAs: 0
2016-05-17 22:37:46,091 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524665833_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:37:46,091 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524665833_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:46,091 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524665833_0001_00_000001
2016-05-17 22:37:46,091 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524665833_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:37:46,091 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:46,092 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ield'_RESULTS.0_VALUES_-1, applicationId=application_1463524665833_0001
2016-05-17 22:37:46,094 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:46,095 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:46,095 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:46,095 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:46,095 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:46,095 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:46,096 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:46,096 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:46,096 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:46,096 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:46,097 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:46,097 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:46,097 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:46,103 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/D5A6C8B36B6746A4A9A3D483FB674C4F/E12572A8897E456598FAB90BC985B2A7/.tez/application_1463524665833_0001
2016-05-17 22:37:46,104 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-17 22:37:46,104 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...ield'_RESULTS.0_VALUES_-1
2016-05-17 22:37:46,105 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-17 22:37:46,105 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_-1/_temporary
2016-05-17 22:37:46,106 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.291
2016-05-17 22:37:46,106 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_-1
2016-05-17 22:37:46,141 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:46,141 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:46,177 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-17 22:37:46,180 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-17 22:37:46,180 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:46,180 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:46,181 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_'field2'
2016-05-17 22:37:46,201 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:46,202 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:46,255 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 22:37:46,269 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-17 22:37:46,270 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:46,270 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:46,270 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:46,292 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:46,293 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS
2016-05-17 22:37:46,307 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:46,307 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:46,307 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS"]
2016-05-17 22:37:46,308 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:46,308 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:46,308 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:46,309 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-17 22:37:46,309 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:46,327 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:46,333 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:46,333 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:46,333 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:46,346 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DCB98C03F39B49679725E9CB998E2DFB/CFE93885242845C684E36AD8321B3ADD/.tez/application_1463524666333_0001 doesn't exist and is created
2016-05-17 22:37:46,371 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:46,371 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:46,372 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DCB98C03F39B49679725E9CB998E2DFB/CFE93885242845C684E36AD8321B3ADD/.tez/application_1463524666333_0001
2016-05-17 22:37:46,377 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524666333_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:46,393 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:46,394 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:46,398 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:46,398 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:46,398 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DCB98C03F39B49679725E9CB998E2DFB/CFE93885242845C684E36AD8321B3ADD/.tez/application_1463524666333_0001/localmode-log-dir
2016-05-17 22:37:46,399 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DCB98C03F39B49679725E9CB998E2DFB/CFE93885242845C684E36AD8321B3ADD/.tez/application_1463524666333_0001/localmode-log-dir/history.txt.appattempt_1463524666333_0001_000000, maxErrors=10
2016-05-17 22:37:46,399 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:46,400 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524666333_0001_000000, appSubmitTime=1463524666376, launchTime=1463524666377
2016-05-17 22:37:46,404 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:46,406 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:46,409 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:46,409 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:46,410 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:42162
2016-05-17 22:37:46,413 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:46,413 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:46,413 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:46,415 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524666333_0001_000000, startTime=1463524666415
2016-05-17 22:37:46,415 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:46,415 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:46,472 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:46,472 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:46,472 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ..._RESULTS.0_VALUES_RESULTS, applicationId=application_1463524666333_0001, dagName=[DCB98C03F39B49679725E9CB998E2DFB/CFE93885242845C684E36AD8321B3ADD] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-17 22:37:46,488 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [DCB98C03F39B49679725E9CB998E2DFB/CFE93885242845C684E36AD8321B3ADD] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-17 22:37:46,489 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524666333_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DCB98C03F39B49679725E9CB998E2DFB/CFE93885242845C684E36AD8321B3ADD/.tez/application_1463524666333_0001/localmode-log-dir/dag_1463524666333_0001_1.dot
2016-05-17 22:37:46,490 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DCB98C03F39B49679725E9CB998E2DFB/CFE93885242845C684E36AD8321B3ADD/.tez/application_1463524666333_0001/localmode-log-dir/dag_1463524666333_0001_1-tez-dag.pb.txt
2016-05-17 22:37:46,494 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524666333_0001_1
2016-05-17 22:37:46,494 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:46,494 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [DCB98C03F39B49679725E9CB998E2DFB/CFE93885242845C684E36AD8321B3ADD] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-17 22:37:46 Running Dag: dag_1463524666333_0001_1
2016-05-17 22:37:46,505 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666333_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524666333_0001_1, submitTime=1463524666488
2016-05-17 22:37:46,505 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:46,505 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B83BD6615654431BBE12335E614B2006
2016-05-17 22:37:46,506 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:46,506 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666333_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524666333_0001_1, initTime=1463524666505
2016-05-17 22:37:46,506 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:46,506 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524666333_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:46,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666333_0001_1][Event:DAG_STARTED]: dagID=dag_1463524666333_0001_1, startTime=1463524666507
2016-05-17 22:37:46,507 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:46,507 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:46,507 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524666333_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:46,507 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B83BD6615654431BBE12335E614B2006 : {7C944004DD364083BA6A1B9F625863C7={InputName=7C944004DD364083BA6A1B9F625863C7}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:46,507 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 7C944004DD364083BA6A1B9F625863C7, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:46,507 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006]
2016-05-17 22:37:46,508 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006]
2016-05-17 22:37:46,508 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006]
2016-05-17 22:37:46,508 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006]. Starting root input initializers: 1
2016-05-17 22:37:46,508 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:46,508 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 7C944004DD364083BA6A1B9F625863C7 on vertex vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006]
2016-05-17 22:37:46,509 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ..._RESULTS.0_VALUES_RESULTS, applicationId=application_1463524666333_0001, dagName=[DCB98C03F39B49679725E9CB998E2DFB/CFE93885242845C684E36AD8321B3ADD] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-17 22:37:46,511 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:46,511 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 7C944004DD364083BA6A1B9F625863C7 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:46,512 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:46,512 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:46,513 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:46,514 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 7C944004DD364083BA6A1B9F625863C7 on vertex vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006]
2016-05-17 22:37:46,514 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:46,514 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {7C944004DD364083BA6A1B9F625863C7=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:46,514 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006] parallelism set to 1
2016-05-17 22:37:46,514 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:46,515 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666333_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524666333_0001_1_00, eventCount=1
2016-05-17 22:37:46,515 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006]
2016-05-17 22:37:46,515 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=620DB14F70B04EEF82C74ECC6AC153F5, vertexId=vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:46,515 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=620DB14F70B04EEF82C74ECC6AC153F5, vertexId=vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006]
2016-05-17 22:37:46,517 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=620DB14F70B04EEF82C74ECC6AC153F5, vertexName=B83BD6615654431BBE12335E614B2006, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:46,517 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=620DB14F70B04EEF82C74ECC6AC153F5, vertexId=vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006]
2016-05-17 22:37:46,519 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:46,519 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666333_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B83BD6615654431BBE12335E614B2006, vertexId=vertex_1463524666333_0001_1_00, initRequestedTime=1463524666507, initedTime=1463524666519, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:46,519 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006] with distanceFromRoot: 0
2016-05-17 22:37:46,519 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:46,519 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B83BD6615654431BBE12335E614B2006
2016-05-17 22:37:46,519 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006]
2016-05-17 22:37:46,520 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:46,520 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666333_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524666333_0001_1_00, startRequestedTime=1463524666519, startedTime=1463524666519
2016-05-17 22:37:46,520 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:46,520 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:46,521 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666333_0001_1][Event:TASK_STARTED]: vertexName=B83BD6615654431BBE12335E614B2006, taskId=task_1463524666333_0001_1_00_000000, scheduledTime=1463524666520, launchTime=1463524666520
2016-05-17 22:37:46,521 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524666333_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:46,521 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006]
2016-05-17 22:37:46,521 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [DCB98C03F39B49679725E9CB998E2DFB/CFE93885242845C684E36AD8321B3ADD] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS, VertexName: B83BD6615654431BBE12335E614B2006, VertexParallelism: 1, TaskAttemptID:attempt_1463524666333_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7C944004DD364083BA6A1B9F625863C7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=620DB14F70B04EEF82C74ECC6AC153F5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:46,522 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524666333_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:46,522 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:46,523 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:46,523 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:46,523 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:46,524 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524666333_0001_00_000001
2016-05-17 22:37:46,524 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524666333_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:46,525 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524666333_0001_00_000001
2016-05-17 22:37:46,525 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:46,526 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524666333_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:46,526 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:46,526 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666333_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524666333_0001_00_000001, launchTime=1463524666526
2016-05-17 22:37:46,527 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524666333_0001_00_000001
2016-05-17 22:37:46,527 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524666333_0001_1_00_000000_0] to container: [container_1463524666333_0001_00_000001]
2016-05-17 22:37:46,527 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524666333_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:46,528 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524666333_0001_00_000001 given task: attempt_1463524666333_0001_1_00_000000_0
2016-05-17 22:37:46,528 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524666333_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524666333_0001_1_00_000000_0
2016-05-17 22:37:46,528 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524666333_0001_1_00_000000_0] started. Is using containerId: [container_1463524666333_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:46,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666333_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B83BD6615654431BBE12335E614B2006, taskAttemptId=attempt_1463524666333_0001_1_00_000000_0, startTime=1463524666528, containerId=container_1463524666333_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524666333_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:46 Starting to run new task attempt: attempt_1463524666333_0001_1_00_000000_0
2016-05-17 22:37:46,528 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524666333_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:46,529 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524666333_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:46,529 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:46,528 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524666333_0001_1_00_000000_0
2016-05-17 22:37:46,529 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:46,529 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:46,529 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:46,530 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:46,530 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:46,530 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:46,530 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DCB98C03F39B49679725E9CB998E2DFB/CFE93885242845C684E36AD8321B3ADD] every.every/(1/1) ..._RESULTS.0_VALUES_RESULTS, VertexName: B83BD6615654431BBE12335E614B2006, VertexParallelism: 1, TaskAttemptID:attempt_1463524666333_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7C944004DD364083BA6A1B9F625863C7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=620DB14F70B04EEF82C74ECC6AC153F5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:46,530 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:46,531 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524666333_0001_1_00_000000_0
2016-05-17 22:37:46,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:46,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:46,531 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524666333_0001, with dag id: dag_1463524666333_0001_1
2016-05-17 22:37:46,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=7C944004DD364083BA6A1B9F625863C7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:46,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:46,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:46,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 7C944004DD364083BA6A1B9F625863C7
2016-05-17 22:37:46,532 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 7C944004DD364083BA6A1B9F625863C7
2016-05-17 22:37:46,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=620DB14F70B04EEF82C74ECC6AC153F5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:46,534 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:46,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 620DB14F70B04EEF82C74ECC6AC153F5
2016-05-17 22:37:46,535 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:46,535 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:46,535 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524666333_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:46,535 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:46,536 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 620DB14F70B04EEF82C74ECC6AC153F5
2016-05-17 22:37:46,535 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 7C944004DD364083BA6A1B9F625863C7
2016-05-17 22:37:46,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 7C944004DD364083BA6A1B9F625863C7
2016-05-17 22:37:46,544 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B83BD6615654431BBE12335E614B2006, ordinal: 0
2016-05-17 22:37:46,544 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B83BD6615654431BBE12335E614B2006, mem on start (mb), free: 294, total: 386, max: 672
2016-05-17 22:37:46,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:46,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:46,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:46,548 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:46,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 620DB14F70B04EEF82C74ECC6AC153F5
2016-05-17 22:37:46,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:46,548 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:46,549 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:46,549 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:46,549 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:46,549 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:46,549 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:46,549 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:46,549 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 7C944004DD364083BA6A1B9F625863C7, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:46,550 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 620DB14F70B04EEF82C74ECC6AC153F5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:46,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:46,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 7C944004DD364083BA6A1B9F625863C7 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:46,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 7C944004DD364083BA6A1B9F625863C7
2016-05-17 22:37:46,550 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:46,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 7C944004DD364083BA6A1B9F625863C7
2016-05-17 22:37:46,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:46,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:46,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:46,551 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524666333_0001_1_00_000000_0
2016-05-17 22:37:46,552 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:46,553 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1b4a1b5
2016-05-17 22:37:46,554 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1b4a1b5
2016-05-17 22:37:46,554 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:46,554 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:46,554 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:46,555 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 7C944004DD364083BA6A1B9F625863C7
2016-05-17 22:37:46,555 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS"], id: 620DB14F70B04EEF82C74ECC6AC153F5
2016-05-17 22:37:46,556 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS"]
2016-05-17 22:37:46,556 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:46,556 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:46,556 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B83BD6615654431BBE12335E614B2006, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:46,557 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:46,561 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:46,564 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:46,568 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:46,569 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:46,569 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524666333_0001_1_00_000000_0
2016-05-17 22:37:46,569 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524666333_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:46,570 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524666333_0001_1_00_000000_0
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:46,575 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524666557, Process_Duration=13, Process_End_Time=1463524666570, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:37:46,576 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B83BD6615654431BBE12335E614B2006, mem on close (mb), free: 291, total: 386, max: 672
2016-05-17 22:37:46,576 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666333_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B83BD6615654431BBE12335E614B2006, taskAttemptId=attempt_1463524666333_0001_1_00_000000_0, startTime=1463524666528, finishTime=1463524666575, timeTaken=47, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524666557, Process_Duration=13, Process_End_Time=1463524666570, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:37:46,576 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:46,576 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524666333_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:37:46,577 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524666333_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:37:46,577 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666333_0001_1][Event:TASK_FINISHED]: vertexName=B83BD6615654431BBE12335E614B2006, taskId=task_1463524666333_0001_1_00_000000, startTime=1463524666528, finishTime=1463524666577, timeTaken=49, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
], counters=Counters: 0
2016-05-17 22:37:46,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:46,577 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524666333_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:37:46,579 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006] : 1
2016-05-17 22:37:46,579 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:46,579 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006] because task failed: task_1463524666333_0001_1_00_000000
2016-05-17 22:37:46,584 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:37:46,584 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:37:46,584 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006]
2016-05-17 22:37:46,584 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=620DB14F70B04EEF82C74ECC6AC153F5, vertexId=vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006]
2016-05-17 22:37:46,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=B83BD6615654431BBE12335E614B2006, index=0
2016-05-17 22:37:46,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=B83BD6615654431BBE12335E614B2006, sourceVertex=7C944004DD364083BA6A1B9F625863C7
2016-05-17 22:37:46,586 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=B83BD6615654431BBE12335E614B2006, sourceVertex=620DB14F70B04EEF82C74ECC6AC153F5
2016-05-17 22:37:46,586 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666333_0001_1][Event:VERTEX_FINISHED]: vertexName=B83BD6615654431BBE12335E614B2006, vertexId=vertex_1463524666333_0001_1_00, initRequestedTime=1463524666507, initedTime=1463524666519, startRequestedTime=1463524666519, startedTime=1463524666519, finishTime=1463524666585, timeTaken=66, status=FAILED, diagnostics=Task failed, taskId=task_1463524666333_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524666528, firstTasksToStart=[ task_1463524666333_0001_1_00_000000 ], lastTaskFinishTime=1463524666575, lastTasksToFinish=[ task_1463524666333_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:37:46,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:46,586 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524666333_0001_00_000001
2016-05-17 22:37:46,588 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524666333_0001_00_000001
2016-05-17 22:37:46,588 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:37:46,589 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:46,589 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666333_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524666333_0001_00_000001, stoppedTime=1463524666588, exitStatus=0
2016-05-17 22:37:46,589 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524666333_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:46,589 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524666333_0001_1_00_000000_0 failed: true container: container_1463524666333_0001_00_000001 numFailedTAs: 0
2016-05-17 22:37:46,589 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:37:46,589 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:37:46,589 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524666333_0001_1
2016-05-17 22:37:46,590 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 620DB14F70B04EEF82C74ECC6AC153F5 for vertex: vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006]
2016-05-17 22:37:46,589 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524666333_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:37:46,590 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524666333_0001_00_000001 completed but with errors
2016-05-17 22:37:46,591 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524666333_0001_1, queueSize=0
2016-05-17 22:37:46,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666333_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524666333_0001_1, startTime=1463524666507, finishTime=1463524666590, timeTaken=83, status=FAILED, diagnostics=Vertex failed, vertexName=B83BD6615654431BBE12335E614B2006, vertexId=vertex_1463524666333_0001_1_00, diagnostics=[Task failed, taskId=task_1463524666333_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524666333_0001_1_00 [B83BD6615654431BBE12335E614B2006] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:37:46,592 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524666333_0001_1 finished with state: FAILED
2016-05-17 22:37:46,595 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524666333_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:46,595 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524666333_0001_00_000001
2016-05-17 22:37:46,595 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524666333_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:37:46,594 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:46,596 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524666333_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:46 Completed Dag: dag_1463524666333_0001_1
2016-05-17 22:37:46,596 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524666333_0001_1, dagState=FAILED
2016-05-17 22:37:46,596 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524666333_0001_1_post
2016-05-17 22:37:46,596 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:46,597 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:46,597 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524666333_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:37:46,608 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ..._RESULTS.0_VALUES_RESULTS, applicationId=application_1463524666333_0001
2016-05-17 22:37:46,610 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:46,610 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:46,611 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:46,611 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:46,611 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:46,611 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:46,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:46,612 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:46,612 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:46,612 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:46,613 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:46,613 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:46,613 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:46,618 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DCB98C03F39B49679725E9CB998E2DFB/CFE93885242845C684E36AD8321B3ADD/.tez/application_1463524666333_0001
2016-05-17 22:37:46,618 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-17 22:37:46,621 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ..._RESULTS.0_VALUES_RESULTS
2016-05-17 22:37:46,622 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-17 22:37:46,622 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_RESULTS/_temporary
2016-05-17 22:37:46,622 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.315
2016-05-17 22:37:46,623 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_RESULTS
2016-05-17 22:37:46,645 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:46,645 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:46,677 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 22:37:46,684 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 22:37:46,685 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:46,685 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:46,685 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:46,700 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:46,701 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL
2016-05-17 22:37:46,710 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:46,711 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:46,711 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL"]
2016-05-17 22:37:46,711 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:46,711 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:46,711 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:46,712 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-17 22:37:46,712 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:46,724 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:46,728 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:46,729 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:46,729 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:46,739 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/50F2E078F3B64EEAA4001B20F60069F2/DBB9E10AD75245DCA992445072B0EA6B/.tez/application_1463524666728_0001 doesn't exist and is created
2016-05-17 22:37:46,755 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:46,755 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:46,756 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/50F2E078F3B64EEAA4001B20F60069F2/DBB9E10AD75245DCA992445072B0EA6B/.tez/application_1463524666728_0001
2016-05-17 22:37:46,759 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524666728_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:46,770 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:46,771 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:46,772 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:46,772 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:46,772 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/50F2E078F3B64EEAA4001B20F60069F2/DBB9E10AD75245DCA992445072B0EA6B/.tez/application_1463524666728_0001/localmode-log-dir
2016-05-17 22:37:46,773 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/50F2E078F3B64EEAA4001B20F60069F2/DBB9E10AD75245DCA992445072B0EA6B/.tez/application_1463524666728_0001/localmode-log-dir/history.txt.appattempt_1463524666728_0001_000000, maxErrors=10
2016-05-17 22:37:46,773 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:46,773 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524666728_0001_000000, appSubmitTime=1463524666759, launchTime=1463524666759
2016-05-17 22:37:46,775 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:46,778 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:46,780 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:46,780 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:46,782 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:60183
2016-05-17 22:37:46,784 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:46,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524666728_0001_000000, startTime=1463524666786
2016-05-17 22:37:46,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:46,786 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:46,828 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:46,828 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:46,855 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:46,856 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:46,856 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_VALUES_ALL, applicationId=application_1463524666728_0001, dagName=[50F2E078F3B64EEAA4001B20F60069F2/DBB9E10AD75245DCA992445072B0EA6B] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-17 22:37:46,870 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [50F2E078F3B64EEAA4001B20F60069F2/DBB9E10AD75245DCA992445072B0EA6B] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-17 22:37:46,871 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524666728_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/50F2E078F3B64EEAA4001B20F60069F2/DBB9E10AD75245DCA992445072B0EA6B/.tez/application_1463524666728_0001/localmode-log-dir/dag_1463524666728_0001_1.dot
2016-05-17 22:37:46,871 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/50F2E078F3B64EEAA4001B20F60069F2/DBB9E10AD75245DCA992445072B0EA6B/.tez/application_1463524666728_0001/localmode-log-dir/dag_1463524666728_0001_1-tez-dag.pb.txt
2016-05-17 22:37:46,875 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524666728_0001_1
2016-05-17 22:37:46,875 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:46,875 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [50F2E078F3B64EEAA4001B20F60069F2/DBB9E10AD75245DCA992445072B0EA6B] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-17 22:37:46 Running Dag: dag_1463524666728_0001_1
2016-05-17 22:37:46,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666728_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524666728_0001_1, submitTime=1463524666870
2016-05-17 22:37:46,887 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:46,888 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B142C667984E49CF805E3385447FB66E
2016-05-17 22:37:46,888 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:46,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666728_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524666728_0001_1, initTime=1463524666887
2016-05-17 22:37:46,888 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:46,889 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524666728_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:46,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666728_0001_1][Event:DAG_STARTED]: dagID=dag_1463524666728_0001_1, startTime=1463524666889
2016-05-17 22:37:46,889 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:46,890 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524666728_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:46,890 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B142C667984E49CF805E3385447FB66E : {F4B48D48B39A451EA629755EF519C611={InputName=F4B48D48B39A451EA629755EF519C611}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:46,890 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F4B48D48B39A451EA629755EF519C611, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:46,890 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E]
2016-05-17 22:37:46,890 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E]
2016-05-17 22:37:46,890 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E]
2016-05-17 22:37:46,891 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E]. Starting root input initializers: 1
2016-05-17 22:37:46,891 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:46,891 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F4B48D48B39A451EA629755EF519C611 on vertex vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E]
2016-05-17 22:37:46,889 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:46,890 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...eld'_RESULTS.0_VALUES_ALL, applicationId=application_1463524666728_0001, dagName=[50F2E078F3B64EEAA4001B20F60069F2/DBB9E10AD75245DCA992445072B0EA6B] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-17 22:37:46,893 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:46,894 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F4B48D48B39A451EA629755EF519C611 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:46,894 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:46,895 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:46,896 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:46,896 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F4B48D48B39A451EA629755EF519C611 on vertex vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E]
2016-05-17 22:37:46,897 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:46,897 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F4B48D48B39A451EA629755EF519C611=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:46,897 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E] parallelism set to 1
2016-05-17 22:37:46,897 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:46,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666728_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524666728_0001_1_00, eventCount=1
2016-05-17 22:37:46,898 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E]
2016-05-17 22:37:46,898 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1FB72355B99D4CADBF804667F9B82BC8, vertexId=vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:46,898 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1FB72355B99D4CADBF804667F9B82BC8, vertexId=vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E]
2016-05-17 22:37:46,900 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1FB72355B99D4CADBF804667F9B82BC8, vertexName=B142C667984E49CF805E3385447FB66E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:46,900 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1FB72355B99D4CADBF804667F9B82BC8, vertexId=vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E]
2016-05-17 22:37:46,901 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666728_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B142C667984E49CF805E3385447FB66E, vertexId=vertex_1463524666728_0001_1_00, initRequestedTime=1463524666890, initedTime=1463524666901, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:46,902 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E] with distanceFromRoot: 0
2016-05-17 22:37:46,901 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:46,902 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:46,902 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B142C667984E49CF805E3385447FB66E
2016-05-17 22:37:46,903 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E]
2016-05-17 22:37:46,903 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:46,903 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666728_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524666728_0001_1_00, startRequestedTime=1463524666902, startedTime=1463524666902
2016-05-17 22:37:46,903 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:46,904 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:46,904 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666728_0001_1][Event:TASK_STARTED]: vertexName=B142C667984E49CF805E3385447FB66E, taskId=task_1463524666728_0001_1_00_000000, scheduledTime=1463524666904, launchTime=1463524666904
2016-05-17 22:37:46,904 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524666728_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:46,904 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E]
2016-05-17 22:37:46,905 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [50F2E078F3B64EEAA4001B20F60069F2/DBB9E10AD75245DCA992445072B0EA6B] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL, VertexName: B142C667984E49CF805E3385447FB66E, VertexParallelism: 1, TaskAttemptID:attempt_1463524666728_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F4B48D48B39A451EA629755EF519C611, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1FB72355B99D4CADBF804667F9B82BC8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:46,905 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524666728_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:46,905 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:46,906 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:46,906 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:46,907 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:46,907 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524666728_0001_00_000001
2016-05-17 22:37:46,907 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524666728_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:46,908 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524666728_0001, with dag id: dag_1463524666728_0001_1
2016-05-17 22:37:46,909 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524666728_0001_00_000001
2016-05-17 22:37:46,910 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:46,911 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524666728_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:46,911 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:46,911 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666728_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524666728_0001_00_000001, launchTime=1463524666911
2016-05-17 22:37:46,913 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524666728_0001_00_000001
2016-05-17 22:37:46,914 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524666728_0001_1_00_000000_0] to container: [container_1463524666728_0001_00_000001]
2016-05-17 22:37:46,914 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524666728_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:46,914 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524666728_0001_00_000001 given task: attempt_1463524666728_0001_1_00_000000_0
2016-05-17 22:37:46,914 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524666728_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524666728_0001_1_00_000000_0
2016-05-17 22:37:46,914 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524666728_0001_1_00_000000_0] started. Is using containerId: [container_1463524666728_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:46 Starting to run new task attempt: attempt_1463524666728_0001_1_00_000000_0
2016-05-17 22:37:46,915 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524666728_0001_1_00_000000_0
2016-05-17 22:37:46,915 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:46,915 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:46,915 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:46,915 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:46,915 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:46,915 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:46,915 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666728_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B142C667984E49CF805E3385447FB66E, taskAttemptId=attempt_1463524666728_0001_1_00_000000_0, startTime=1463524666914, containerId=container_1463524666728_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524666728_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:46,915 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:46,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [50F2E078F3B64EEAA4001B20F60069F2/DBB9E10AD75245DCA992445072B0EA6B] every.every/(1/1) ...eld'_RESULTS.0_VALUES_ALL, VertexName: B142C667984E49CF805E3385447FB66E, VertexParallelism: 1, TaskAttemptID:attempt_1463524666728_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F4B48D48B39A451EA629755EF519C611, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1FB72355B99D4CADBF804667F9B82BC8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:46,916 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524666728_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:46,916 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:46,916 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524666728_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:46,917 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524666728_0001_1_00_000000_0
2016-05-17 22:37:46,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:46,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:46,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:46,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F4B48D48B39A451EA629755EF519C611, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:46,919 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524666728_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:46,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:46,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F4B48D48B39A451EA629755EF519C611
2016-05-17 22:37:46,919 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F4B48D48B39A451EA629755EF519C611
2016-05-17 22:37:46,920 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:46,921 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:46,921 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F4B48D48B39A451EA629755EF519C611
2016-05-17 22:37:46,921 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F4B48D48B39A451EA629755EF519C611
2016-05-17 22:37:46,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1FB72355B99D4CADBF804667F9B82BC8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:46,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:46,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1FB72355B99D4CADBF804667F9B82BC8
2016-05-17 22:37:46,925 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:46,926 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1FB72355B99D4CADBF804667F9B82BC8
2016-05-17 22:37:46,932 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B142C667984E49CF805E3385447FB66E, ordinal: 0
2016-05-17 22:37:46,932 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B142C667984E49CF805E3385447FB66E, mem on start (mb), free: 231, total: 386, max: 672
2016-05-17 22:37:46,932 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:46,932 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:46,932 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:46,938 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:46,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1FB72355B99D4CADBF804667F9B82BC8
2016-05-17 22:37:46,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:46,938 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:46,938 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:46,938 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:46,938 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:46,939 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:46,939 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:46,939 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:46,939 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F4B48D48B39A451EA629755EF519C611, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:46,939 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1FB72355B99D4CADBF804667F9B82BC8, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:46,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:46,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F4B48D48B39A451EA629755EF519C611 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:46,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:46,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:46,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F4B48D48B39A451EA629755EF519C611
2016-05-17 22:37:46,940 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:46,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F4B48D48B39A451EA629755EF519C611
2016-05-17 22:37:46,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:46,941 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524666728_0001_1_00_000000_0
2016-05-17 22:37:46,941 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:46,942 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6d87a9d6
2016-05-17 22:37:46,942 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6d87a9d6
2016-05-17 22:37:46,943 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:46,943 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:46,944 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:46,945 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F4B48D48B39A451EA629755EF519C611
2016-05-17 22:37:46,945 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL"], id: 1FB72355B99D4CADBF804667F9B82BC8
2016-05-17 22:37:46,945 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL"]
2016-05-17 22:37:46,945 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:46,946 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:46,946 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B142C667984E49CF805E3385447FB66E, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:46,946 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:46,947 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:46,949 ERROR element.SourceStage (SourceStage.java:map(118)) - caught throwable
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:46,950 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:46,950 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:46,951 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524666728_0001_1_00_000000_0
2016-05-17 22:37:46,951 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524666728_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:46,951 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524666728_0001_1_00_000000_0
cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
2016-05-17 22:37:46,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 18 [[File System Counters FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=1044, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1463524666946, Process_Duration=5, Process_End_Time=1463524666951, Read_Duration=0, Tuples_Read=1][cascading.flow.StepCounters Tuples_Read=1]]
2016-05-17 22:37:46,957 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666728_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B142C667984E49CF805E3385447FB66E, taskAttemptId=attempt_1463524666728_0001_1_00_000000_0, startTime=1463524666914, finishTime=1463524666956, timeTaken=42, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=51, FILE_BYTES_WRITTEN=1044, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524666946, Process_Duration=5, Process_End_Time=1463524666951, Read_Duration=0, Tuples_Read=1, cascading.flow.StepCounters, Tuples_Read=1
2016-05-17 22:37:46,957 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:46,957 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524666728_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:37:46,957 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524666728_0001_1_00_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:37:46,958 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666728_0001_1][Event:TASK_FINISHED]: vertexName=B142C667984E49CF805E3385447FB66E, taskId=task_1463524666728_0001_1_00_000000, startTime=1463524666914, finishTime=1463524666957, timeTaken=43, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
], counters=Counters: 0
2016-05-17 22:37:46,958 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:46,957 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B142C667984E49CF805E3385447FB66E, mem on close (mb), free: 229, total: 386, max: 672
2016-05-17 22:37:46,958 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524666728_0001_1_00_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:37:46,959 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E] : 1
2016-05-17 22:37:46,959 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:46,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=B142C667984E49CF805E3385447FB66E, index=0
2016-05-17 22:37:46,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=B142C667984E49CF805E3385447FB66E, sourceVertex=F4B48D48B39A451EA629755EF519C611
2016-05-17 22:37:46,959 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E] because task failed: task_1463524666728_0001_1_00_000000
2016-05-17 22:37:46,961 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:37:46,961 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:37:46,961 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E]
2016-05-17 22:37:46,961 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=1FB72355B99D4CADBF804667F9B82BC8, vertexId=vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E]
2016-05-17 22:37:46,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=B142C667984E49CF805E3385447FB66E, sourceVertex=1FB72355B99D4CADBF804667F9B82BC8
2016-05-17 22:37:46,962 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524666728_0001_00_000001
2016-05-17 22:37:46,962 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524666728_0001_00_000001
2016-05-17 22:37:46,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666728_0001_1][Event:VERTEX_FINISHED]: vertexName=B142C667984E49CF805E3385447FB66E, vertexId=vertex_1463524666728_0001_1_00, initRequestedTime=1463524666890, initedTime=1463524666901, startRequestedTime=1463524666902, startedTime=1463524666902, finishTime=1463524666962, timeTaken=60, status=FAILED, diagnostics=Task failed, taskId=task_1463524666728_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524666914, firstTasksToStart=[ task_1463524666728_0001_1_00_000000 ], lastTaskFinishTime=1463524666956, lastTasksToFinish=[ task_1463524666728_0001_1_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:37:46,963 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:37:46,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666728_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524666728_0001_00_000001, stoppedTime=1463524666963, exitStatus=0
2016-05-17 22:37:46,963 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524666728_0001_00_000001 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:46,964 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524666728_0001_1_00_000000_0 failed: true container: container_1463524666728_0001_00_000001 numFailedTAs: 0
2016-05-17 22:37:46,964 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=1
2016-05-17 22:37:46,964 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:37:46,964 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524666728_0001_1
2016-05-17 22:37:46,964 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 1FB72355B99D4CADBF804667F9B82BC8 for vertex: vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E]
2016-05-17 22:37:46,965 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524666728_0001_1, queueSize=0
2016-05-17 22:37:46,965 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524666728_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524666728_0001_1, startTime=1463524666889, finishTime=1463524666964, timeTaken=75, status=FAILED, diagnostics=Vertex failed, vertexName=B142C667984E49CF805E3385447FB66E, vertexId=vertex_1463524666728_0001_1_00, diagnostics=[Task failed, taskId=task_1463524666728_0001_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every.every][cascading.detail.EachAssemblyFactory.createAssembly(EachAssemblyFactory.java:33)] operator Each failed executing operation
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:108)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.FunctionEachStage$1.collect(FunctionEachStage.java:81)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:95)
at cascading.TestFunction.operate(TestFunction.java:104)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:40)
at cascading.flow.stream.element.SourceStage.map(SourceStage.java:110)
at cascading.flow.stream.element.SourceStage.run(SourceStage.java:66)
at cascading.flow.tez.stream.element.TezSourceStage.run(TezSourceStage.java:95)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestFunction.operate(TestFunction.java:101)
at cascading.flow.stream.element.FunctionEachStage.receive(FunctionEachStage.java:100)
... 23 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524666728_0001_1_00 [B142C667984E49CF805E3385447FB66E] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 2, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, TOTAL_LAUNCHED_TASKS=1
2016-05-17 22:37:46,965 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524666728_0001_1 finished with state: FAILED
2016-05-17 22:37:46,965 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524666728_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:46,966 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524666728_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:37:46 Completed Dag: dag_1463524666728_0001_1
2016-05-17 22:37:46,966 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524666728_0001_1, dagState=FAILED
2016-05-17 22:37:46,966 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524666728_0001_1_post
2016-05-17 22:37:46,963 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:46,967 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:46,967 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:46,967 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:46,968 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:46,968 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524666728_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:37:46,968 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524666728_0001_00_000001 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:37:46,968 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524666728_0001_00_000001 completed but with errors
2016-05-17 22:37:46,969 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524666728_0001_00_000001 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:37:46,969 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524666728_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:37:46,969 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524666728_0001_1_00_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:37:46,977 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...eld'_RESULTS.0_VALUES_ALL, applicationId=application_1463524666728_0001
2016-05-17 22:37:46,979 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:46,979 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:46,979 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:46,979 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:46,979 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:46,983 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:46,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:46,983 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:46,983 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:46,984 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:46,984 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:46,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:46,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:46,988 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/50F2E078F3B64EEAA4001B20F60069F2/DBB9E10AD75245DCA992445072B0EA6B/.tez/application_1463524666728_0001
2016-05-17 22:37:46,988 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping all jobs
2016-05-17 22:37:46,988 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopping: (1/1) ...eld'_RESULTS.0_VALUES_ALL
2016-05-17 22:37:46,989 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] stopped all jobs
2016-05-17 22:37:46,989 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_VALUES_ALL/_temporary
2016-05-17 22:37:46,992 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.279
2016-05-17 22:37:46,992 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_VALUES_ALL
2016-05-17 22:37:47,012 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:47,013 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:47,041 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 22:37:47,051 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-17 22:37:47,051 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:47,051 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:47,052 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:47,065 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:47,065 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1
2016-05-17 22:37:47,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:47,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:47,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1"]
2016-05-17 22:37:47,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:47,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:47,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:47,076 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-17 22:37:47,076 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:47,088 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:47,092 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:47,092 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:47,093 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:47,101 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/99CB9A395EB845429F8BD3D16F2304D1/F23BFCCAB4224F6B826B2AD18B6B0820/.tez/application_1463524667092_0001 doesn't exist and is created
2016-05-17 22:37:47,118 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:47,118 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:47,119 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/99CB9A395EB845429F8BD3D16F2304D1/F23BFCCAB4224F6B826B2AD18B6B0820/.tez/application_1463524667092_0001
2016-05-17 22:37:47,122 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524667092_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:47,132 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:47,133 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:47,134 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:47,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:47,135 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/99CB9A395EB845429F8BD3D16F2304D1/F23BFCCAB4224F6B826B2AD18B6B0820/.tez/application_1463524667092_0001/localmode-log-dir
2016-05-17 22:37:47,135 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/99CB9A395EB845429F8BD3D16F2304D1/F23BFCCAB4224F6B826B2AD18B6B0820/.tez/application_1463524667092_0001/localmode-log-dir/history.txt.appattempt_1463524667092_0001_000000, maxErrors=10
2016-05-17 22:37:47,135 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:47,135 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524667092_0001_000000, appSubmitTime=1463524667122, launchTime=1463524667122
2016-05-17 22:37:47,139 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:47,141 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:47,142 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55854
2016-05-17 22:37:47,143 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:47,143 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:47,144 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:47,146 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524667092_0001_000000, startTime=1463524667146
2016-05-17 22:37:47,146 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:47,146 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:47,219 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:47,219 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:47,219 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...'field'_RESULTS.0_ARGS_-1, applicationId=application_1463524667092_0001, dagName=[99CB9A395EB845429F8BD3D16F2304D1/F23BFCCAB4224F6B826B2AD18B6B0820] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-17 22:37:47,234 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [99CB9A395EB845429F8BD3D16F2304D1/F23BFCCAB4224F6B826B2AD18B6B0820] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-17 22:37:47,235 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524667092_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/99CB9A395EB845429F8BD3D16F2304D1/F23BFCCAB4224F6B826B2AD18B6B0820/.tez/application_1463524667092_0001/localmode-log-dir/dag_1463524667092_0001_1.dot
2016-05-17 22:37:47,235 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/99CB9A395EB845429F8BD3D16F2304D1/F23BFCCAB4224F6B826B2AD18B6B0820/.tez/application_1463524667092_0001/localmode-log-dir/dag_1463524667092_0001_1-tez-dag.pb.txt
2016-05-17 22:37:47,240 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524667092_0001_1
2016-05-17 22:37:47,240 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:47,240 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [99CB9A395EB845429F8BD3D16F2304D1/F23BFCCAB4224F6B826B2AD18B6B0820] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-17 22:37:47 Running Dag: dag_1463524667092_0001_1
2016-05-17 22:37:47,251 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:47,252 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667092_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524667092_0001_1, submitTime=1463524667234
2016-05-17 22:37:47,252 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A8169729656B4264AFDC6BC66D4688C9
2016-05-17 22:37:47,252 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:47,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:47,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667092_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524667092_0001_1, initTime=1463524667252
2016-05-17 22:37:47,253 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524667092_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:47,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667092_0001_1][Event:DAG_STARTED]: dagID=dag_1463524667092_0001_1, startTime=1463524667253
2016-05-17 22:37:47,253 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:47,253 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524667092_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:47,253 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A8169729656B4264AFDC6BC66D4688C9 : {7B72D9B60DAD48E4AF5E56904FF86A4D={InputName=7B72D9B60DAD48E4AF5E56904FF86A4D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:47,253 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 7B72D9B60DAD48E4AF5E56904FF86A4D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:47,254 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9]
2016-05-17 22:37:47,254 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9]
2016-05-17 22:37:47,254 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9]
2016-05-17 22:37:47,254 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9]. Starting root input initializers: 1
2016-05-17 22:37:47,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:47,254 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...'field'_RESULTS.0_ARGS_-1, applicationId=application_1463524667092_0001, dagName=[99CB9A395EB845429F8BD3D16F2304D1/F23BFCCAB4224F6B826B2AD18B6B0820] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1
2016-05-17 22:37:47,255 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:47,256 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 7B72D9B60DAD48E4AF5E56904FF86A4D on vertex vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9]
2016-05-17 22:37:47,257 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:47,257 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 7B72D9B60DAD48E4AF5E56904FF86A4D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:47,258 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:47,258 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:47,259 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:47,259 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 7B72D9B60DAD48E4AF5E56904FF86A4D on vertex vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9]
2016-05-17 22:37:47,260 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:47,260 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {7B72D9B60DAD48E4AF5E56904FF86A4D=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:47,260 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9] parallelism set to 1
2016-05-17 22:37:47,260 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:47,260 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667092_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524667092_0001_1_00, eventCount=1
2016-05-17 22:37:47,260 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9]
2016-05-17 22:37:47,260 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0670315136B44BCFB83B17335CD48D79, vertexId=vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:47,261 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0670315136B44BCFB83B17335CD48D79, vertexId=vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9]
2016-05-17 22:37:47,262 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0670315136B44BCFB83B17335CD48D79, vertexName=A8169729656B4264AFDC6BC66D4688C9, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:47,262 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0670315136B44BCFB83B17335CD48D79, vertexId=vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9]
2016-05-17 22:37:47,263 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667092_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A8169729656B4264AFDC6BC66D4688C9, vertexId=vertex_1463524667092_0001_1_00, initRequestedTime=1463524667253, initedTime=1463524667263, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:47,263 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:47,263 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9] with distanceFromRoot: 0
2016-05-17 22:37:47,264 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:47,264 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A8169729656B4264AFDC6BC66D4688C9
2016-05-17 22:37:47,264 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9]
2016-05-17 22:37:47,264 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:47,264 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667092_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524667092_0001_1_00, startRequestedTime=1463524667264, startedTime=1463524667264
2016-05-17 22:37:47,265 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:47,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:47,265 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667092_0001_1][Event:TASK_STARTED]: vertexName=A8169729656B4264AFDC6BC66D4688C9, taskId=task_1463524667092_0001_1_00_000000, scheduledTime=1463524667265, launchTime=1463524667265
2016-05-17 22:37:47,266 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524667092_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:47,266 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9]
2016-05-17 22:37:47,266 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [99CB9A395EB845429F8BD3D16F2304D1/F23BFCCAB4224F6B826B2AD18B6B0820] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1, VertexName: A8169729656B4264AFDC6BC66D4688C9, VertexParallelism: 1, TaskAttemptID:attempt_1463524667092_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7B72D9B60DAD48E4AF5E56904FF86A4D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0670315136B44BCFB83B17335CD48D79, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:47,266 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524667092_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:47,267 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:47,267 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:47,268 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:47,268 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:47,268 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524667092_0001_00_000001
2016-05-17 22:37:47,268 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524667092_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:47,270 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524667092_0001_00_000001
2016-05-17 22:37:47,270 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:47,271 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524667092_0001, with dag id: dag_1463524667092_0001_1
2016-05-17 22:37:47,271 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524667092_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:47,271 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:47,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667092_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524667092_0001_00_000001, launchTime=1463524667271
2016-05-17 22:37:47,272 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524667092_0001_00_000001
2016-05-17 22:37:47,273 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524667092_0001_1_00_000000_0] to container: [container_1463524667092_0001_00_000001]
2016-05-17 22:37:47,274 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524667092_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:47,275 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524667092_0001_1_00_000000_0] started. Is using containerId: [container_1463524667092_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:47,275 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667092_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A8169729656B4264AFDC6BC66D4688C9, taskAttemptId=attempt_1463524667092_0001_1_00_000000_0, startTime=1463524667274, containerId=container_1463524667092_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524667092_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:47,275 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524667092_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:47,275 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:47,275 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524667092_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:47,276 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524667092_0001_00_000001 given task: attempt_1463524667092_0001_1_00_000000_0
2016-05-17 22:37:47,276 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524667092_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524667092_0001_1_00_000000_0
2016-05-17 22:37:47 Starting to run new task attempt: attempt_1463524667092_0001_1_00_000000_0
2016-05-17 22:37:47,278 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524667092_0001_1_00_000000_0
2016-05-17 22:37:47,278 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:47,278 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:47,278 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:47,279 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:47,279 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:47,279 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:47,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [99CB9A395EB845429F8BD3D16F2304D1/F23BFCCAB4224F6B826B2AD18B6B0820] every.every/(1/1) ...'field'_RESULTS.0_ARGS_-1, VertexName: A8169729656B4264AFDC6BC66D4688C9, VertexParallelism: 1, TaskAttemptID:attempt_1463524667092_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7B72D9B60DAD48E4AF5E56904FF86A4D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0670315136B44BCFB83B17335CD48D79, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:47,279 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:47,280 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524667092_0001_1_00_000000_0
2016-05-17 22:37:47,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:47,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:47,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=7B72D9B60DAD48E4AF5E56904FF86A4D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:47,281 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524667092_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:47,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0670315136B44BCFB83B17335CD48D79, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:47,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:47,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:47,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:47,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 7B72D9B60DAD48E4AF5E56904FF86A4D
2016-05-17 22:37:47,283 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 7B72D9B60DAD48E4AF5E56904FF86A4D
2016-05-17 22:37:47,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0670315136B44BCFB83B17335CD48D79
2016-05-17 22:37:47,283 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:47,284 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0670315136B44BCFB83B17335CD48D79
2016-05-17 22:37:47,290 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:47,290 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 7B72D9B60DAD48E4AF5E56904FF86A4D
2016-05-17 22:37:47,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 7B72D9B60DAD48E4AF5E56904FF86A4D
2016-05-17 22:37:47,292 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:47,293 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:47,293 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:47,297 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:47,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0670315136B44BCFB83B17335CD48D79
2016-05-17 22:37:47,300 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A8169729656B4264AFDC6BC66D4688C9, ordinal: 0
2016-05-17 22:37:47,300 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A8169729656B4264AFDC6BC66D4688C9, mem on start (mb), free: 168, total: 386, max: 672
2016-05-17 22:37:47,300 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:47,300 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:47,300 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:47,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:47,301 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:47,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:47,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:47,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:47,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:47,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:47,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:47,301 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 7B72D9B60DAD48E4AF5E56904FF86A4D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:47,302 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0670315136B44BCFB83B17335CD48D79, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:47,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:47,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 7B72D9B60DAD48E4AF5E56904FF86A4D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:47,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 7B72D9B60DAD48E4AF5E56904FF86A4D
2016-05-17 22:37:47,302 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:47,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 7B72D9B60DAD48E4AF5E56904FF86A4D
2016-05-17 22:37:47,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:47,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:47,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:47,303 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524667092_0001_1_00_000000_0
2016-05-17 22:37:47,303 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:47,304 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2937c534
2016-05-17 22:37:47,305 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2937c534
2016-05-17 22:37:47,305 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:47,306 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:47,307 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:47,308 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 7B72D9B60DAD48E4AF5E56904FF86A4D
2016-05-17 22:37:47,308 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1"], id: 0670315136B44BCFB83B17335CD48D79
2016-05-17 22:37:47,308 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1"]
2016-05-17 22:37:47,308 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:47,308 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:47,309 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A8169729656B4264AFDC6BC66D4688C9, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:47,311 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:47,311 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:47,312 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524667092_0001_1_00_000000_0
2016-05-17 22:37:47,312 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524667092_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:47,312 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524667092_0001_1_00_000000_0
2016-05-17 22:37:47,312 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A8169729656B4264AFDC6BC66D4688C9, mem on close (mb), free: 166, total: 386, max: 672
2016-05-17 22:37:47 Completed running task attempt: attempt_1463524667092_0001_1_00_000000_0
2016-05-17 22:37:47,313 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524667092_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:47,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524667309, Process_Duration=3, Process_End_Time=1463524667312, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:47,314 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:47,314 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:47,314 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667092_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A8169729656B4264AFDC6BC66D4688C9, taskAttemptId=attempt_1463524667092_0001_1_00_000000_0, startTime=1463524667274, finishTime=1463524667314, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524667309, Process_Duration=3, Process_End_Time=1463524667312, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:47,314 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524667092_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:47,314 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:47,315 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:47,314 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524667092_0001_00_000001
2016-05-17 22:37:47,314 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524667092_0001_1_00_000000_0
2016-05-17 22:37:47,316 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:47,316 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667092_0001_1][Event:TASK_FINISHED]: vertexName=A8169729656B4264AFDC6BC66D4688C9, taskId=task_1463524667092_0001_1_00_000000, startTime=1463524667274, finishTime=1463524667316, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463524667092_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524667309, Process_Duration=3, Process_End_Time=1463524667312, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:47,316 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524667092_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:47,317 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9] : 1
2016-05-17 22:37:47,317 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9]
2016-05-17 22:37:47,318 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:47,318 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667092_0001_1][Event:VERTEX_FINISHED]: vertexName=A8169729656B4264AFDC6BC66D4688C9, vertexId=vertex_1463524667092_0001_1_00, initRequestedTime=1463524667253, initedTime=1463524667263, startRequestedTime=1463524667264, startedTime=1463524667264, finishTime=1463524667317, timeTaken=53, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524667309, Process_Duration=3, Process_End_Time=1463524667312, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524667274, firstTasksToStart=[ task_1463524667092_0001_1_00_000000 ], lastTaskFinishTime=1463524667314, lastTasksToFinish=[ task_1463524667092_0001_1_00_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524667092_0001_1_00_000000 ], longestDurationTasks=[ task_1463524667092_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:47,318 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:47,318 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524667092_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:47,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667092_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524667092_0001_00_000001, stoppedTime=1463524667319, exitStatus=0
2016-05-17 22:37:47,319 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524667092_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:47,319 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:47,319 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524667092_0001_1
2016-05-17 22:37:47,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667092_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524667092_0001_1
2016-05-17 22:37:47,319 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:47,319 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0670315136B44BCFB83B17335CD48D79 for vertex: vertex_1463524667092_0001_1_00 [A8169729656B4264AFDC6BC66D4688C9]
2016-05-17 22:37:47,327 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524667092_0001_1, queueSize=0
2016-05-17 22:37:47,328 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524667092_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:47,328 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:47,328 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524667092_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:47,328 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524667092_0001_00_000001, Exiting
2016-05-17 22:37:47,328 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524667092_0001_00_000001
2016-05-17 22:37:47,328 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524667092_0001_00_000001
2016-05-17 22:37:47,329 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667092_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524667092_0001_1, startTime=1463524667253, finishTime=1463524667327, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=405274624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524667309, Process_Duration=3, Process_End_Time=1463524667312, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:47,329 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524667092_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:47,329 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524667092_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:47,329 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524667092_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:47,330 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524667092_0001_00_000001
2016-05-17 22:37:47,330 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524667092_0001_00_000001 completed successfully
2016-05-17 22:37:47 Completed Dag: dag_1463524667092_0001_1
2016-05-17 22:37:47,331 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524667092_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:47,331 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524667092_0001_1_post
2016-05-17 22:37:47,332 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:47,332 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:47,332 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524667092_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:37:47,333 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524667092_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:37:47,337 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...'field'_RESULTS.0_ARGS_-1, applicationId=application_1463524667092_0001
2016-05-17 22:37:47,339 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:47,339 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:47,339 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:47,339 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:47,339 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:47,340 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:47,340 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:47,340 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:47,340 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:47,341 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:47,341 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:47,341 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:47,341 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:47,345 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/99CB9A395EB845429F8BD3D16F2304D1/F23BFCCAB4224F6B826B2AD18B6B0820/.tez/application_1463524667092_0001
2016-05-17 22:37:47,345 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_-1/_temporary
2016-05-17 22:37:47,346 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.270
2016-05-17 22:37:47,410 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_-1
2016-05-17 22:37:47,430 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:47,430 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:47,458 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 22:37:47,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-17 22:37:47,462 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:47,463 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:47,463 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_'field2'
2016-05-17 22:37:47,503 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:47,504 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:47,539 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 22:37:47,545 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 22:37:47,546 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:47,546 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:47,546 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:47,565 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:47,566 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS
2016-05-17 22:37:47,579 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:47,580 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:47,580 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS"]
2016-05-17 22:37:47,580 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:47,580 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:47,582 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:47,583 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-17 22:37:47,583 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:47,602 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:47,608 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:47,608 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:47,609 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:47,618 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/F9A3C52BEA41443C84D31A88B85218D7/BFB3406E9CAB4915992725505C2DAF22/.tez/application_1463524667608_0001 doesn't exist and is created
2016-05-17 22:37:47,638 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:47,638 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:47,638 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/F9A3C52BEA41443C84D31A88B85218D7/BFB3406E9CAB4915992725505C2DAF22/.tez/application_1463524667608_0001
2016-05-17 22:37:47,641 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524667608_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:47,651 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:47,651 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:47,653 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:47,653 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:47,653 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/F9A3C52BEA41443C84D31A88B85218D7/BFB3406E9CAB4915992725505C2DAF22/.tez/application_1463524667608_0001/localmode-log-dir
2016-05-17 22:37:47,653 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/F9A3C52BEA41443C84D31A88B85218D7/BFB3406E9CAB4915992725505C2DAF22/.tez/application_1463524667608_0001/localmode-log-dir/history.txt.appattempt_1463524667608_0001_000000, maxErrors=10
2016-05-17 22:37:47,653 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:47,654 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524667608_0001_000000, appSubmitTime=1463524667641, launchTime=1463524667641
2016-05-17 22:37:47,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:47,663 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49893
2016-05-17 22:37:47,665 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:47,669 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:47,669 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:47,671 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:47,672 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524667608_0001_000000, startTime=1463524667672
2016-05-17 22:37:47,672 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:47,672 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:47,738 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:47,738 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:47,739 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_ARGS_RESULTS, applicationId=application_1463524667608_0001, dagName=[F9A3C52BEA41443C84D31A88B85218D7/BFB3406E9CAB4915992725505C2DAF22] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-17 22:37:47,753 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [F9A3C52BEA41443C84D31A88B85218D7/BFB3406E9CAB4915992725505C2DAF22] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-17 22:37:47,754 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524667608_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/F9A3C52BEA41443C84D31A88B85218D7/BFB3406E9CAB4915992725505C2DAF22/.tez/application_1463524667608_0001/localmode-log-dir/dag_1463524667608_0001_1.dot
2016-05-17 22:37:47,755 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/F9A3C52BEA41443C84D31A88B85218D7/BFB3406E9CAB4915992725505C2DAF22/.tez/application_1463524667608_0001/localmode-log-dir/dag_1463524667608_0001_1-tez-dag.pb.txt
2016-05-17 22:37:47,759 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524667608_0001_1
2016-05-17 22:37:47,759 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:47,759 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [F9A3C52BEA41443C84D31A88B85218D7/BFB3406E9CAB4915992725505C2DAF22] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-17 22:37:47 Running Dag: dag_1463524667608_0001_1
2016-05-17 22:37:47,772 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667608_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524667608_0001_1, submitTime=1463524667753
2016-05-17 22:37:47,772 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:47,773 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DD385BA643A545F0ADB7DA53DEC51E03
2016-05-17 22:37:47,773 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:47,774 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667608_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524667608_0001_1, initTime=1463524667772
2016-05-17 22:37:47,774 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:47,774 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524667608_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:47,775 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667608_0001_1][Event:DAG_STARTED]: dagID=dag_1463524667608_0001_1, startTime=1463524667774
2016-05-17 22:37:47,775 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:47,775 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524667608_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:47,775 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DD385BA643A545F0ADB7DA53DEC51E03 : {2DAD2761B9E44CA8AB650AD0707D78FA={InputName=2DAD2761B9E44CA8AB650AD0707D78FA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:47,775 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2DAD2761B9E44CA8AB650AD0707D78FA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:47,775 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03]
2016-05-17 22:37:47,775 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03]
2016-05-17 22:37:47,776 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03]
2016-05-17 22:37:47,776 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03]. Starting root input initializers: 1
2016-05-17 22:37:47,776 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:47,777 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2DAD2761B9E44CA8AB650AD0707D78FA on vertex vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03]
2016-05-17 22:37:47,775 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:47,777 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.0_ARGS_RESULTS, applicationId=application_1463524667608_0001, dagName=[F9A3C52BEA41443C84D31A88B85218D7/BFB3406E9CAB4915992725505C2DAF22] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS
2016-05-17 22:37:47,778 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:47,778 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2DAD2761B9E44CA8AB650AD0707D78FA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:47,779 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:47,779 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:47,780 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:47,780 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2DAD2761B9E44CA8AB650AD0707D78FA on vertex vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03]
2016-05-17 22:37:47,781 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:47,781 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2DAD2761B9E44CA8AB650AD0707D78FA=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:47,781 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03] parallelism set to 1
2016-05-17 22:37:47,781 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:47,781 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667608_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524667608_0001_1_00, eventCount=1
2016-05-17 22:37:47,781 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03]
2016-05-17 22:37:47,781 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0A2E1E182E41432EB25F72763FF35F4E, vertexId=vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:47,782 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0A2E1E182E41432EB25F72763FF35F4E, vertexId=vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03]
2016-05-17 22:37:47,783 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0A2E1E182E41432EB25F72763FF35F4E, vertexName=DD385BA643A545F0ADB7DA53DEC51E03, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:47,783 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0A2E1E182E41432EB25F72763FF35F4E, vertexId=vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03]
2016-05-17 22:37:47,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667608_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DD385BA643A545F0ADB7DA53DEC51E03, vertexId=vertex_1463524667608_0001_1_00, initRequestedTime=1463524667775, initedTime=1463524667784, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:47,784 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03] with distanceFromRoot: 0
2016-05-17 22:37:47,785 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:47,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:47,785 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DD385BA643A545F0ADB7DA53DEC51E03
2016-05-17 22:37:47,785 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03]
2016-05-17 22:37:47,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:47,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667608_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524667608_0001_1_00, startRequestedTime=1463524667785, startedTime=1463524667785
2016-05-17 22:37:47,786 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:47,787 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:47,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667608_0001_1][Event:TASK_STARTED]: vertexName=DD385BA643A545F0ADB7DA53DEC51E03, taskId=task_1463524667608_0001_1_00_000000, scheduledTime=1463524667787, launchTime=1463524667787
2016-05-17 22:37:47,787 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524667608_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:47,787 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03]
2016-05-17 22:37:47,788 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [F9A3C52BEA41443C84D31A88B85218D7/BFB3406E9CAB4915992725505C2DAF22] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS, VertexName: DD385BA643A545F0ADB7DA53DEC51E03, VertexParallelism: 1, TaskAttemptID:attempt_1463524667608_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2DAD2761B9E44CA8AB650AD0707D78FA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0A2E1E182E41432EB25F72763FF35F4E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:47,788 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524667608_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:47,788 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:47,789 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:47,789 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:47,789 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:47,790 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524667608_0001_00_000001
2016-05-17 22:37:47,790 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524667608_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:47,792 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524667608_0001_00_000001
2016-05-17 22:37:47,792 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:47,793 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524667608_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:47,793 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:47,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667608_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524667608_0001_00_000001, launchTime=1463524667793
2016-05-17 22:37:47,794 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524667608_0001_00_000001
2016-05-17 22:37:47,795 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524667608_0001_1_00_000000_0] to container: [container_1463524667608_0001_00_000001]
2016-05-17 22:37:47,795 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524667608_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:47,795 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524667608_0001_00_000001 given task: attempt_1463524667608_0001_1_00_000000_0
2016-05-17 22:37:47,795 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524667608_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524667608_0001_1_00_000000_0
2016-05-17 22:37:47,795 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524667608_0001_1_00_000000_0] started. Is using containerId: [container_1463524667608_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:47 Starting to run new task attempt: attempt_1463524667608_0001_1_00_000000_0
2016-05-17 22:37:47,796 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667608_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DD385BA643A545F0ADB7DA53DEC51E03, taskAttemptId=attempt_1463524667608_0001_1_00_000000_0, startTime=1463524667795, containerId=container_1463524667608_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524667608_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:47,796 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524667608_0001_1_00_000000_0
2016-05-17 22:37:47,796 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:47,796 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524667608_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:47,796 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524667608_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:47,797 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:47,797 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:47,797 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:47,797 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:47,797 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:47,798 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:47,798 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F9A3C52BEA41443C84D31A88B85218D7/BFB3406E9CAB4915992725505C2DAF22] every.every/(1/1) ...d'_RESULTS.0_ARGS_RESULTS, VertexName: DD385BA643A545F0ADB7DA53DEC51E03, VertexParallelism: 1, TaskAttemptID:attempt_1463524667608_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2DAD2761B9E44CA8AB650AD0707D78FA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=0A2E1E182E41432EB25F72763FF35F4E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:47,798 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524667608_0001, with dag id: dag_1463524667608_0001_1
2016-05-17 22:37:47,798 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:47,799 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524667608_0001_1_00_000000_0
2016-05-17 22:37:47,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:47,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:47,800 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:47,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2DAD2761B9E44CA8AB650AD0707D78FA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:47,802 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524667608_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:47,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0A2E1E182E41432EB25F72763FF35F4E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:47,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:47,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0A2E1E182E41432EB25F72763FF35F4E
2016-05-17 22:37:47,803 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:47,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:47,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2DAD2761B9E44CA8AB650AD0707D78FA
2016-05-17 22:37:47,803 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2DAD2761B9E44CA8AB650AD0707D78FA
2016-05-17 22:37:47,803 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0A2E1E182E41432EB25F72763FF35F4E
2016-05-17 22:37:47,807 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:47,811 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:47,811 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2DAD2761B9E44CA8AB650AD0707D78FA
2016-05-17 22:37:47,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2DAD2761B9E44CA8AB650AD0707D78FA
2016-05-17 22:37:47,815 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DD385BA643A545F0ADB7DA53DEC51E03, ordinal: 0
2016-05-17 22:37:47,815 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD385BA643A545F0ADB7DA53DEC51E03, mem on start (mb), free: 262, total: 368, max: 672
2016-05-17 22:37:47,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:47,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:47,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:47,820 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:47,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0A2E1E182E41432EB25F72763FF35F4E
2016-05-17 22:37:47,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:47,821 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:47,821 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:47,821 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:47,822 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:47,822 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:47,822 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:47,823 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:47,823 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2DAD2761B9E44CA8AB650AD0707D78FA, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:47,823 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0A2E1E182E41432EB25F72763FF35F4E, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:47,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:47,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2DAD2761B9E44CA8AB650AD0707D78FA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:47,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:47,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:47,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2DAD2761B9E44CA8AB650AD0707D78FA
2016-05-17 22:37:47,825 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:47,825 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2DAD2761B9E44CA8AB650AD0707D78FA
2016-05-17 22:37:47,825 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:47,825 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524667608_0001_1_00_000000_0
2016-05-17 22:37:47,827 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:47,828 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7c310d79
2016-05-17 22:37:47,829 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:47,829 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7c310d79
2016-05-17 22:37:47,830 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2DAD2761B9E44CA8AB650AD0707D78FA
2016-05-17 22:37:47,830 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS"], id: 0A2E1E182E41432EB25F72763FF35F4E
2016-05-17 22:37:47,831 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:47,831 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:47,832 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS"]
2016-05-17 22:37:47,832 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:47,832 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:47,833 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DD385BA643A545F0ADB7DA53DEC51E03, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:47,836 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:47,836 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:47,837 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524667608_0001_1_00_000000_0
2016-05-17 22:37:47,837 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524667608_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:47,837 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524667608_0001_1_00_000000_0
2016-05-17 22:37:47,837 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD385BA643A545F0ADB7DA53DEC51E03, mem on close (mb), free: 260, total: 368, max: 672
2016-05-17 22:37:47 Completed running task attempt: attempt_1463524667608_0001_1_00_000000_0
2016-05-17 22:37:47,838 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524667608_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:47,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524667833, Process_Duration=4, Process_End_Time=1463524667837, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:47,840 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667608_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DD385BA643A545F0ADB7DA53DEC51E03, taskAttemptId=attempt_1463524667608_0001_1_00_000000_0, startTime=1463524667795, finishTime=1463524667840, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524667833, Process_Duration=4, Process_End_Time=1463524667837, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:47,840 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:47,840 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524667608_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:47,841 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524667608_0001_1_00_000000_0
2016-05-17 22:37:47,840 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:47,841 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:47,841 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:47,841 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667608_0001_1][Event:TASK_FINISHED]: vertexName=DD385BA643A545F0ADB7DA53DEC51E03, taskId=task_1463524667608_0001_1_00_000000, startTime=1463524667795, finishTime=1463524667841, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1463524667608_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524667833, Process_Duration=4, Process_End_Time=1463524667837, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:47,842 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524667608_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:47,842 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03] : 1
2016-05-17 22:37:47,842 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03]
2016-05-17 22:37:47,842 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524667608_0001_00_000001
2016-05-17 22:37:47,842 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:47,843 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:47,845 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667608_0001_1][Event:VERTEX_FINISHED]: vertexName=DD385BA643A545F0ADB7DA53DEC51E03, vertexId=vertex_1463524667608_0001_1_00, initRequestedTime=1463524667775, initedTime=1463524667784, startRequestedTime=1463524667785, startedTime=1463524667785, finishTime=1463524667842, timeTaken=57, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524667833, Process_Duration=4, Process_End_Time=1463524667837, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524667795, firstTasksToStart=[ task_1463524667608_0001_1_00_000000 ], lastTaskFinishTime=1463524667840, lastTasksToFinish=[ task_1463524667608_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524667608_0001_1_00_000000 ], longestDurationTasks=[ task_1463524667608_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:47,845 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:47,846 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524667608_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:47,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667608_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524667608_0001_00_000001, stoppedTime=1463524667846, exitStatus=0
2016-05-17 22:37:47,846 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524667608_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:47,846 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:47,846 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524667608_0001_1
2016-05-17 22:37:47,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667608_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524667608_0001_1
2016-05-17 22:37:47,847 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0A2E1E182E41432EB25F72763FF35F4E for vertex: vertex_1463524667608_0001_1_00 [DD385BA643A545F0ADB7DA53DEC51E03]
2016-05-17 22:37:47,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:47,852 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524667608_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:47,853 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524667608_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:47,853 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524667608_0001_00_000001, Exiting
2016-05-17 22:37:47,853 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524667608_0001_00_000001
2016-05-17 22:37:47,853 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524667608_0001_00_000001
2016-05-17 22:37:47,854 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524667608_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:47,854 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524667608_0001_00_000001 completed successfully
2016-05-17 22:37:47,856 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524667608_0001_1, queueSize=0
2016-05-17 22:37:47,856 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:47,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524667608_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524667608_0001_1, startTime=1463524667774, finishTime=1463524667855, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524667833, Process_Duration=4, Process_End_Time=1463524667837, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:47,857 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524667608_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:47,857 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524667608_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:47,857 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524667608_0001_00_000001
2016-05-17 22:37:47,857 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524667608_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:47 Completed Dag: dag_1463524667608_0001_1
2016-05-17 22:37:47,859 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524667608_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:47,859 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524667608_0001_1_post
2016-05-17 22:37:47,859 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:47,860 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:47,863 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.0_ARGS_RESULTS, applicationId=application_1463524667608_0001
2016-05-17 22:37:47,864 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:47,865 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:47,865 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:47,865 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:47,865 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:47,865 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:47,865 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:47,865 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:47,867 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:47,867 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:47,867 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:47,868 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:47,868 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:47,872 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/F9A3C52BEA41443C84D31A88B85218D7/BFB3406E9CAB4915992725505C2DAF22/.tez/application_1463524667608_0001
2016-05-17 22:37:47,872 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.0_ARGS_RESULTS/_temporary
2016-05-17 22:37:47,873 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.293
2016-05-17 22:37:47,893 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:47,893 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:47,941 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_RESULTS
2016-05-17 22:37:47,960 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:47,960 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:47,986 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-17 22:37:47,987 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-17 22:37:47,987 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:47,988 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:47,988 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.0_ARGS_ALL
2016-05-17 22:37:48,008 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:48,008 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:48,038 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 22:37:48,045 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 22:37:48,046 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:48,046 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:48,046 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:48,059 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:48,060 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1
2016-05-17 22:37:48,069 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:48,069 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:48,070 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1"]
2016-05-17 22:37:48,070 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:48,070 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:48,070 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:48,071 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-17 22:37:48,071 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:48,083 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:48,087 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:48,087 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:48,088 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:48,097 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/686D3F6371EA44928AB7545DE5ED4457/CB5AAFB321E04355909121EDCE128F95/.tez/application_1463524668087_0001 doesn't exist and is created
2016-05-17 22:37:48,113 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:48,113 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:48,114 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/686D3F6371EA44928AB7545DE5ED4457/CB5AAFB321E04355909121EDCE128F95/.tez/application_1463524668087_0001
2016-05-17 22:37:48,117 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524668087_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:48,127 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:48,127 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:48,129 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:48,129 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:48,129 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/686D3F6371EA44928AB7545DE5ED4457/CB5AAFB321E04355909121EDCE128F95/.tez/application_1463524668087_0001/localmode-log-dir
2016-05-17 22:37:48,130 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/686D3F6371EA44928AB7545DE5ED4457/CB5AAFB321E04355909121EDCE128F95/.tez/application_1463524668087_0001/localmode-log-dir/history.txt.appattempt_1463524668087_0001_000000, maxErrors=10
2016-05-17 22:37:48,130 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:48,130 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524668087_0001_000000, appSubmitTime=1463524668117, launchTime=1463524668117
2016-05-17 22:37:48,135 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:48,136 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:40512
2016-05-17 22:37:48,139 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:48,140 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:48,141 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:48,141 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:48,144 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524668087_0001_000000, startTime=1463524668143
2016-05-17 22:37:48,144 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:48,144 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:48,214 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:48,214 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:48,214 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...'_RESULTS.ALL_'field2'_-1, applicationId=application_1463524668087_0001, dagName=[686D3F6371EA44928AB7545DE5ED4457/CB5AAFB321E04355909121EDCE128F95] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-17 22:37:48,228 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [686D3F6371EA44928AB7545DE5ED4457/CB5AAFB321E04355909121EDCE128F95] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-17 22:37:48,229 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524668087_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/686D3F6371EA44928AB7545DE5ED4457/CB5AAFB321E04355909121EDCE128F95/.tez/application_1463524668087_0001/localmode-log-dir/dag_1463524668087_0001_1.dot
2016-05-17 22:37:48,230 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/686D3F6371EA44928AB7545DE5ED4457/CB5AAFB321E04355909121EDCE128F95/.tez/application_1463524668087_0001/localmode-log-dir/dag_1463524668087_0001_1-tez-dag.pb.txt
2016-05-17 22:37:48,236 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524668087_0001_1
2016-05-17 22:37:48,236 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:48,236 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [686D3F6371EA44928AB7545DE5ED4457/CB5AAFB321E04355909121EDCE128F95] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-17 22:37:48 Running Dag: dag_1463524668087_0001_1
2016-05-17 22:37:48,248 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668087_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524668087_0001_1, submitTime=1463524668228
2016-05-17 22:37:48,248 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:48,248 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E7DC419F03444BEFB6FC71CD6C6FEE28
2016-05-17 22:37:48,249 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:48,249 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668087_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524668087_0001_1, initTime=1463524668248
2016-05-17 22:37:48,249 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:48,249 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524668087_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:48,250 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668087_0001_1][Event:DAG_STARTED]: dagID=dag_1463524668087_0001_1, startTime=1463524668250
2016-05-17 22:37:48,250 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:48,250 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524668087_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:48,250 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E7DC419F03444BEFB6FC71CD6C6FEE28 : {B41C7EEC1FC84C4FB4BABE4A8BFDF157={InputName=B41C7EEC1FC84C4FB4BABE4A8BFDF157}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:48,250 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B41C7EEC1FC84C4FB4BABE4A8BFDF157, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:48,250 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28]
2016-05-17 22:37:48,250 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28]
2016-05-17 22:37:48,251 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28]
2016-05-17 22:37:48,251 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28]. Starting root input initializers: 1
2016-05-17 22:37:48,251 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:48,252 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...'_RESULTS.ALL_'field2'_-1, applicationId=application_1463524668087_0001, dagName=[686D3F6371EA44928AB7545DE5ED4457/CB5AAFB321E04355909121EDCE128F95] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1
2016-05-17 22:37:48,252 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B41C7EEC1FC84C4FB4BABE4A8BFDF157 on vertex vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28]
2016-05-17 22:37:48,250 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:48,255 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:48,255 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B41C7EEC1FC84C4FB4BABE4A8BFDF157 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:48,256 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:48,256 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:48,258 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:48,258 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B41C7EEC1FC84C4FB4BABE4A8BFDF157 on vertex vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28]
2016-05-17 22:37:48,259 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:48,259 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B41C7EEC1FC84C4FB4BABE4A8BFDF157=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:48,259 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28] parallelism set to 1
2016-05-17 22:37:48,259 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:48,260 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668087_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524668087_0001_1_00, eventCount=1
2016-05-17 22:37:48,260 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28]
2016-05-17 22:37:48,260 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1505D19E893947C09404BC8C93AE41A1, vertexId=vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:48,260 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1505D19E893947C09404BC8C93AE41A1, vertexId=vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28]
2016-05-17 22:37:48,262 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1505D19E893947C09404BC8C93AE41A1, vertexName=E7DC419F03444BEFB6FC71CD6C6FEE28, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:48,262 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1505D19E893947C09404BC8C93AE41A1, vertexId=vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28]
2016-05-17 22:37:48,263 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668087_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E7DC419F03444BEFB6FC71CD6C6FEE28, vertexId=vertex_1463524668087_0001_1_00, initRequestedTime=1463524668250, initedTime=1463524668263, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:48,263 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:48,264 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28] with distanceFromRoot: 0
2016-05-17 22:37:48,264 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:48,264 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E7DC419F03444BEFB6FC71CD6C6FEE28
2016-05-17 22:37:48,264 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28]
2016-05-17 22:37:48,265 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668087_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524668087_0001_1_00, startRequestedTime=1463524668264, startedTime=1463524668264
2016-05-17 22:37:48,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:48,265 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:48,266 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668087_0001_1][Event:TASK_STARTED]: vertexName=E7DC419F03444BEFB6FC71CD6C6FEE28, taskId=task_1463524668087_0001_1_00_000000, scheduledTime=1463524668265, launchTime=1463524668265
2016-05-17 22:37:48,266 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:48,266 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524668087_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:48,266 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28]
2016-05-17 22:37:48,267 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [686D3F6371EA44928AB7545DE5ED4457/CB5AAFB321E04355909121EDCE128F95] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1, VertexName: E7DC419F03444BEFB6FC71CD6C6FEE28, VertexParallelism: 1, TaskAttemptID:attempt_1463524668087_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B41C7EEC1FC84C4FB4BABE4A8BFDF157, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1505D19E893947C09404BC8C93AE41A1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:48,267 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524668087_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:48,267 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:48,268 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:48,268 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:48,268 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:48,269 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524668087_0001_00_000001
2016-05-17 22:37:48,269 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524668087_0001, with dag id: dag_1463524668087_0001_1
2016-05-17 22:37:48,269 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668087_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:48,271 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524668087_0001_00_000001
2016-05-17 22:37:48,271 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:48,272 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668087_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:48,272 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668087_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524668087_0001_00_000001, launchTime=1463524668272
2016-05-17 22:37:48,272 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:48,273 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524668087_0001_00_000001
2016-05-17 22:37:48,273 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524668087_0001_1_00_000000_0] to container: [container_1463524668087_0001_00_000001]
2016-05-17 22:37:48,273 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668087_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:48,274 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524668087_0001_00_000001 given task: attempt_1463524668087_0001_1_00_000000_0
2016-05-17 22:37:48,274 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524668087_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524668087_0001_1_00_000000_0
2016-05-17 22:37:48 Starting to run new task attempt: attempt_1463524668087_0001_1_00_000000_0
2016-05-17 22:37:48,275 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524668087_0001_1_00_000000_0
2016-05-17 22:37:48,275 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:48,275 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:48,275 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:48,275 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:48,276 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:48,276 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:48,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [686D3F6371EA44928AB7545DE5ED4457/CB5AAFB321E04355909121EDCE128F95] every.every/(1/1) ...'_RESULTS.ALL_'field2'_-1, VertexName: E7DC419F03444BEFB6FC71CD6C6FEE28, VertexParallelism: 1, TaskAttemptID:attempt_1463524668087_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B41C7EEC1FC84C4FB4BABE4A8BFDF157, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1505D19E893947C09404BC8C93AE41A1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:48,276 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:48,276 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524668087_0001_1_00_000000_0] started. Is using containerId: [container_1463524668087_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:48,277 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524668087_0001_1_00_000000_0
2016-05-17 22:37:48,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:48,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:48,277 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668087_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E7DC419F03444BEFB6FC71CD6C6FEE28, taskAttemptId=attempt_1463524668087_0001_1_00_000000_0, startTime=1463524668276, containerId=container_1463524668087_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524668087_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:48,277 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:48,278 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524668087_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:48,279 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524668087_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:48,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:48,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B41C7EEC1FC84C4FB4BABE4A8BFDF157, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:48,279 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524668087_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:48,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:48,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B41C7EEC1FC84C4FB4BABE4A8BFDF157
2016-05-17 22:37:48,281 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B41C7EEC1FC84C4FB4BABE4A8BFDF157
2016-05-17 22:37:48,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1505D19E893947C09404BC8C93AE41A1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:48,281 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:48,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:48,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1505D19E893947C09404BC8C93AE41A1
2016-05-17 22:37:48,282 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:48,283 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1505D19E893947C09404BC8C93AE41A1
2016-05-17 22:37:48,283 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:48,284 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B41C7EEC1FC84C4FB4BABE4A8BFDF157
2016-05-17 22:37:48,284 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B41C7EEC1FC84C4FB4BABE4A8BFDF157
2016-05-17 22:37:48,291 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E7DC419F03444BEFB6FC71CD6C6FEE28, ordinal: 0
2016-05-17 22:37:48,291 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E7DC419F03444BEFB6FC71CD6C6FEE28, mem on start (mb), free: 166, total: 368, max: 672
2016-05-17 22:37:48,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:48,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:48,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:48,294 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:48,294 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1505D19E893947C09404BC8C93AE41A1
2016-05-17 22:37:48,294 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:48,294 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:48,294 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:48,294 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:48,294 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:48,295 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:48,295 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:48,295 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:48,295 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B41C7EEC1FC84C4FB4BABE4A8BFDF157, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:48,295 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1505D19E893947C09404BC8C93AE41A1, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:48,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:48,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B41C7EEC1FC84C4FB4BABE4A8BFDF157 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:48,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B41C7EEC1FC84C4FB4BABE4A8BFDF157
2016-05-17 22:37:48,295 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:48,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B41C7EEC1FC84C4FB4BABE4A8BFDF157
2016-05-17 22:37:48,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:48,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:48,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:48,296 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524668087_0001_1_00_000000_0
2016-05-17 22:37:48,296 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:48,297 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@557ae1da
2016-05-17 22:37:48,298 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@557ae1da
2016-05-17 22:37:48,298 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:48,298 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:48,299 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:48,301 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B41C7EEC1FC84C4FB4BABE4A8BFDF157
2016-05-17 22:37:48,301 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1"], id: 1505D19E893947C09404BC8C93AE41A1
2016-05-17 22:37:48,301 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1"]
2016-05-17 22:37:48,302 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:48,302 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:48,302 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E7DC419F03444BEFB6FC71CD6C6FEE28, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:48,304 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:48,305 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:48,305 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524668087_0001_1_00_000000_0
2016-05-17 22:37:48,305 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524668087_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:48,306 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524668087_0001_1_00_000000_0
2016-05-17 22:37:48,306 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E7DC419F03444BEFB6FC71CD6C6FEE28, mem on close (mb), free: 165, total: 368, max: 672
2016-05-17 22:37:48 Completed running task attempt: attempt_1463524668087_0001_1_00_000000_0
2016-05-17 22:37:48,307 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524668087_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:48,308 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524668302, Process_Duration=4, Process_End_Time=1463524668306, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:48,308 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:48,308 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:48,308 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668087_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E7DC419F03444BEFB6FC71CD6C6FEE28, taskAttemptId=attempt_1463524668087_0001_1_00_000000_0, startTime=1463524668276, finishTime=1463524668308, timeTaken=32, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524668302, Process_Duration=4, Process_End_Time=1463524668306, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:48,309 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:48,309 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524668087_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:48,309 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524668087_0001_00_000001
2016-05-17 22:37:48,309 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524668087_0001_1_00_000000_0
2016-05-17 22:37:48,309 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:48,310 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:48,310 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668087_0001_1][Event:TASK_FINISHED]: vertexName=E7DC419F03444BEFB6FC71CD6C6FEE28, taskId=task_1463524668087_0001_1_00_000000, startTime=1463524668276, finishTime=1463524668309, timeTaken=33, status=SUCCEEDED, successfulAttemptID=attempt_1463524668087_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524668302, Process_Duration=4, Process_End_Time=1463524668306, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:48,310 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524668087_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:48,310 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28] : 1
2016-05-17 22:37:48,310 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28]
2016-05-17 22:37:48,311 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:48,311 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668087_0001_1][Event:VERTEX_FINISHED]: vertexName=E7DC419F03444BEFB6FC71CD6C6FEE28, vertexId=vertex_1463524668087_0001_1_00, initRequestedTime=1463524668250, initedTime=1463524668263, startRequestedTime=1463524668264, startedTime=1463524668264, finishTime=1463524668310, timeTaken=46, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524668302, Process_Duration=4, Process_End_Time=1463524668306, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524668276, firstTasksToStart=[ task_1463524668087_0001_1_00_000000 ], lastTaskFinishTime=1463524668308, lastTasksToFinish=[ task_1463524668087_0001_1_00_000000 ], minTaskDuration=32, maxTaskDuration=32, avgTaskDuration=32.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524668087_0001_1_00_000000 ], longestDurationTasks=[ task_1463524668087_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:48,311 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:48,311 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668087_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:48,311 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668087_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524668087_0001_00_000001, stoppedTime=1463524668311, exitStatus=0
2016-05-17 22:37:48,311 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668087_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:48,312 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:48,312 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:48,312 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524668087_0001_1
2016-05-17 22:37:48,312 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668087_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524668087_0001_1
2016-05-17 22:37:48,312 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 1505D19E893947C09404BC8C93AE41A1 for vertex: vertex_1463524668087_0001_1_00 [E7DC419F03444BEFB6FC71CD6C6FEE28]
2016-05-17 22:37:48,320 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524668087_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:48,320 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524668087_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:48,320 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524668087_0001_00_000001, Exiting
2016-05-17 22:37:48,320 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524668087_0001_00_000001
2016-05-17 22:37:48,320 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524668087_0001_00_000001
2016-05-17 22:37:48,321 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524668087_0001_1, queueSize=0
2016-05-17 22:37:48,321 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:48,321 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668087_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524668087_0001_1, startTime=1463524668250, finishTime=1463524668319, timeTaken=69, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1188, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524668302, Process_Duration=4, Process_End_Time=1463524668306, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:48,321 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524668087_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:48,321 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524668087_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:48,321 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524668087_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:48 Completed Dag: dag_1463524668087_0001_1
2016-05-17 22:37:48,322 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524668087_0001_00_000001 completed successfully
2016-05-17 22:37:48,322 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524668087_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:48,322 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524668087_0001_00_000001
2016-05-17 22:37:48,323 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524668087_0001_1_post
2016-05-17 22:37:48,323 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:48,323 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:48,323 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668087_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:48,328 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...'_RESULTS.ALL_'field2'_-1, applicationId=application_1463524668087_0001
2016-05-17 22:37:48,329 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:48,329 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:48,330 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:48,330 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:48,330 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:48,330 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:48,331 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:48,331 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:48,331 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:48,331 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:48,331 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:48,332 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:48,332 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:48,335 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/686D3F6371EA44928AB7545DE5ED4457/CB5AAFB321E04355909121EDCE128F95/.tez/application_1463524668087_0001
2016-05-17 22:37:48,336 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_-1/_temporary
2016-05-17 22:37:48,336 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.266
2016-05-17 22:37:48,338 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:48,338 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:48,399 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_-1
2016-05-17 22:37:48,419 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:48,419 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:48,448 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 22:37:48,452 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 22:37:48,452 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:48,452 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:48,453 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:48,466 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:48,467 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'
2016-05-17 22:37:48,475 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:48,475 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:48,476 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'"]
2016-05-17 22:37:48,476 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:48,476 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:48,476 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:48,477 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...LTS.ALL_'field2'_'field2'
2016-05-17 22:37:48,477 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:48,489 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:48,494 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:48,494 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:48,494 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:48,503 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0150EE98CD6244A687E4271692B9C22B/D7706C2282BF4E008EF445DE57D6010C/.tez/application_1463524668494_0001 doesn't exist and is created
2016-05-17 22:37:48,537 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:48,537 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:48,537 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0150EE98CD6244A687E4271692B9C22B/D7706C2282BF4E008EF445DE57D6010C/.tez/application_1463524668494_0001
2016-05-17 22:37:48,540 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524668494_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:48,550 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:48,551 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:48,553 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:48,553 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:48,553 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0150EE98CD6244A687E4271692B9C22B/D7706C2282BF4E008EF445DE57D6010C/.tez/application_1463524668494_0001/localmode-log-dir
2016-05-17 22:37:48,553 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0150EE98CD6244A687E4271692B9C22B/D7706C2282BF4E008EF445DE57D6010C/.tez/application_1463524668494_0001/localmode-log-dir/history.txt.appattempt_1463524668494_0001_000000, maxErrors=10
2016-05-17 22:37:48,553 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:48,554 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524668494_0001_000000, appSubmitTime=1463524668539, launchTime=1463524668539
2016-05-17 22:37:48,557 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:48,561 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:48,562 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:48,562 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:48,565 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49976
2016-05-17 22:37:48,567 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:48,569 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:48,569 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524668494_0001_000000, startTime=1463524668569
2016-05-17 22:37:48,569 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:48,637 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:48,637 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:48,637 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...LTS.ALL_'field2'_'field2', applicationId=application_1463524668494_0001, dagName=[0150EE98CD6244A687E4271692B9C22B/D7706C2282BF4E008EF445DE57D6010C] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-17 22:37:48,652 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [0150EE98CD6244A687E4271692B9C22B/D7706C2282BF4E008EF445DE57D6010C] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-17 22:37:48,654 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524668494_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0150EE98CD6244A687E4271692B9C22B/D7706C2282BF4E008EF445DE57D6010C/.tez/application_1463524668494_0001/localmode-log-dir/dag_1463524668494_0001_1.dot
2016-05-17 22:37:48,654 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0150EE98CD6244A687E4271692B9C22B/D7706C2282BF4E008EF445DE57D6010C/.tez/application_1463524668494_0001/localmode-log-dir/dag_1463524668494_0001_1-tez-dag.pb.txt
2016-05-17 22:37:48,658 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524668494_0001_1
2016-05-17 22:37:48,659 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:48,659 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [0150EE98CD6244A687E4271692B9C22B/D7706C2282BF4E008EF445DE57D6010C] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-17 22:37:48 Running Dag: dag_1463524668494_0001_1
2016-05-17 22:37:48,673 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668494_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524668494_0001_1, submitTime=1463524668653
2016-05-17 22:37:48,673 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:48,673 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B1A8623957734E3BB908D418FD1932A0
2016-05-17 22:37:48,674 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:48,674 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:48,674 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668494_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524668494_0001_1, initTime=1463524668673
2016-05-17 22:37:48,674 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524668494_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:48,675 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668494_0001_1][Event:DAG_STARTED]: dagID=dag_1463524668494_0001_1, startTime=1463524668675
2016-05-17 22:37:48,675 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:48,675 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524668494_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:48,675 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B1A8623957734E3BB908D418FD1932A0 : {9597365D0AA647CC92CB74F41328C918={InputName=9597365D0AA647CC92CB74F41328C918}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:48,675 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9597365D0AA647CC92CB74F41328C918, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:48,676 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0]
2016-05-17 22:37:48,676 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:48,676 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0]
2016-05-17 22:37:48,676 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0]
2016-05-17 22:37:48,676 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...LTS.ALL_'field2'_'field2', applicationId=application_1463524668494_0001, dagName=[0150EE98CD6244A687E4271692B9C22B/D7706C2282BF4E008EF445DE57D6010C] every.every/(1/1) ...LTS.ALL_'field2'_'field2'
2016-05-17 22:37:48,676 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0]. Starting root input initializers: 1
2016-05-17 22:37:48,677 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:48,677 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9597365D0AA647CC92CB74F41328C918 on vertex vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0]
2016-05-17 22:37:48,679 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:48,679 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9597365D0AA647CC92CB74F41328C918 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:48,679 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:48,679 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:48,681 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:48,681 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9597365D0AA647CC92CB74F41328C918 on vertex vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0]
2016-05-17 22:37:48,681 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:48,682 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9597365D0AA647CC92CB74F41328C918=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:48,682 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0] parallelism set to 1
2016-05-17 22:37:48,682 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:48,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668494_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524668494_0001_1_00, eventCount=1
2016-05-17 22:37:48,682 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0]
2016-05-17 22:37:48,682 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=328C7037ED664F2E836047E1F9397E6B, vertexId=vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:48,683 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=328C7037ED664F2E836047E1F9397E6B, vertexId=vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0]
2016-05-17 22:37:48,684 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=328C7037ED664F2E836047E1F9397E6B, vertexName=B1A8623957734E3BB908D418FD1932A0, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:48,685 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=328C7037ED664F2E836047E1F9397E6B, vertexId=vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0]
2016-05-17 22:37:48,686 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:48,686 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668494_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B1A8623957734E3BB908D418FD1932A0, vertexId=vertex_1463524668494_0001_1_00, initRequestedTime=1463524668675, initedTime=1463524668686, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:48,686 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0] with distanceFromRoot: 0
2016-05-17 22:37:48,686 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:48,687 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B1A8623957734E3BB908D418FD1932A0
2016-05-17 22:37:48,687 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0]
2016-05-17 22:37:48,687 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:48,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668494_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524668494_0001_1_00, startRequestedTime=1463524668687, startedTime=1463524668687
2016-05-17 22:37:48,688 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:48,688 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:48,688 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668494_0001_1][Event:TASK_STARTED]: vertexName=B1A8623957734E3BB908D418FD1932A0, taskId=task_1463524668494_0001_1_00_000000, scheduledTime=1463524668688, launchTime=1463524668688
2016-05-17 22:37:48,689 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524668494_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:48,689 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0]
2016-05-17 22:37:48,689 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0150EE98CD6244A687E4271692B9C22B/D7706C2282BF4E008EF445DE57D6010C] every.every/(1/1) ...LTS.ALL_'field2'_'field2', VertexName: B1A8623957734E3BB908D418FD1932A0, VertexParallelism: 1, TaskAttemptID:attempt_1463524668494_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9597365D0AA647CC92CB74F41328C918, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=328C7037ED664F2E836047E1F9397E6B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:48,689 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524668494_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:48,690 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:48,690 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:48,691 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:48,691 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:48,691 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524668494_0001_00_000001
2016-05-17 22:37:48,691 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668494_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:48,693 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524668494_0001_00_000001
2016-05-17 22:37:48,693 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:48,693 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524668494_0001, with dag id: dag_1463524668494_0001_1
2016-05-17 22:37:48,694 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668494_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:48,694 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:48,695 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668494_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524668494_0001_00_000001, launchTime=1463524668694
2016-05-17 22:37:48,695 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524668494_0001_00_000001
2016-05-17 22:37:48,696 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524668494_0001_1_00_000000_0] to container: [container_1463524668494_0001_00_000001]
2016-05-17 22:37:48,696 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668494_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:48,696 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524668494_0001_00_000001 given task: attempt_1463524668494_0001_1_00_000000_0
2016-05-17 22:37:48,696 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524668494_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524668494_0001_1_00_000000_0
2016-05-17 22:37:48,696 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524668494_0001_1_00_000000_0] started. Is using containerId: [container_1463524668494_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:48,697 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668494_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B1A8623957734E3BB908D418FD1932A0, taskAttemptId=attempt_1463524668494_0001_1_00_000000_0, startTime=1463524668696, containerId=container_1463524668494_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524668494_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:48,697 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:48,697 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524668494_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:48 Starting to run new task attempt: attempt_1463524668494_0001_1_00_000000_0
2016-05-17 22:37:48,697 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524668494_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:48,697 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524668494_0001_1_00_000000_0
2016-05-17 22:37:48,698 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:48,698 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:48,698 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:48,698 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:48,698 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:48,698 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:48,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0150EE98CD6244A687E4271692B9C22B/D7706C2282BF4E008EF445DE57D6010C] every.every/(1/1) ...LTS.ALL_'field2'_'field2', VertexName: B1A8623957734E3BB908D418FD1932A0, VertexParallelism: 1, TaskAttemptID:attempt_1463524668494_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9597365D0AA647CC92CB74F41328C918, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=328C7037ED664F2E836047E1F9397E6B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:48,699 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:48,699 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524668494_0001_1_00_000000_0
2016-05-17 22:37:48,700 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524668494_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:48,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:48,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:48,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9597365D0AA647CC92CB74F41328C918, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:48,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=328C7037ED664F2E836047E1F9397E6B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:48,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:48,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:48,702 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:48,702 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9597365D0AA647CC92CB74F41328C918
2016-05-17 22:37:48,702 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9597365D0AA647CC92CB74F41328C918
2016-05-17 22:37:48,702 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 328C7037ED664F2E836047E1F9397E6B
2016-05-17 22:37:48,702 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:48,703 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 328C7037ED664F2E836047E1F9397E6B
2016-05-17 22:37:48,704 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:48,704 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9597365D0AA647CC92CB74F41328C918
2016-05-17 22:37:48,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9597365D0AA647CC92CB74F41328C918
2016-05-17 22:37:48,705 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:48,712 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B1A8623957734E3BB908D418FD1932A0, ordinal: 0
2016-05-17 22:37:48,712 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1A8623957734E3BB908D418FD1932A0, mem on start (mb), free: 278, total: 359, max: 672
2016-05-17 22:37:48,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:48,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:48,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:48,713 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:48,714 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 328C7037ED664F2E836047E1F9397E6B
2016-05-17 22:37:48,714 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:48,714 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:48,714 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:48,714 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:48,715 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:48,715 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:48,715 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:48,715 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:48,716 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9597365D0AA647CC92CB74F41328C918, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:48,716 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 328C7037ED664F2E836047E1F9397E6B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:48,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:48,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9597365D0AA647CC92CB74F41328C918
2016-05-17 22:37:48,716 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:48,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9597365D0AA647CC92CB74F41328C918
2016-05-17 22:37:48,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9597365D0AA647CC92CB74F41328C918 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:48,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:48,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:48,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:48,718 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524668494_0001_1_00_000000_0
2016-05-17 22:37:48,718 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:48,719 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@825d609
2016-05-17 22:37:48,720 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@825d609
2016-05-17 22:37:48,720 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:48,720 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:48,722 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:48,723 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 9597365D0AA647CC92CB74F41328C918
2016-05-17 22:37:48,723 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'"], id: 328C7037ED664F2E836047E1F9397E6B
2016-05-17 22:37:48,723 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'"]
2016-05-17 22:37:48,723 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:48,724 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:48,724 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B1A8623957734E3BB908D418FD1932A0, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:48,726 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:48,726 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:48,727 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524668494_0001_1_00_000000_0
2016-05-17 22:37:48,727 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524668494_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:48,727 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524668494_0001_1_00_000000_0
2016-05-17 22:37:48,728 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1A8623957734E3BB908D418FD1932A0, mem on close (mb), free: 276, total: 359, max: 672
2016-05-17 22:37:48 Completed running task attempt: attempt_1463524668494_0001_1_00_000000_0
2016-05-17 22:37:48,729 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524668494_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:48,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524668724, Process_Duration=3, Process_End_Time=1463524668727, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:48,730 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:48,730 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668494_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B1A8623957734E3BB908D418FD1932A0, taskAttemptId=attempt_1463524668494_0001_1_00_000000_0, startTime=1463524668696, finishTime=1463524668730, timeTaken=34, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524668724, Process_Duration=3, Process_End_Time=1463524668727, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:48,730 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:48,731 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:48,731 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524668494_0001_00_000001
2016-05-17 22:37:48,731 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524668494_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:48,731 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:48,731 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524668494_0001_1_00_000000_0
2016-05-17 22:37:48,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:48,732 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668494_0001_1][Event:TASK_FINISHED]: vertexName=B1A8623957734E3BB908D418FD1932A0, taskId=task_1463524668494_0001_1_00_000000, startTime=1463524668696, finishTime=1463524668732, timeTaken=36, status=SUCCEEDED, successfulAttemptID=attempt_1463524668494_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524668724, Process_Duration=3, Process_End_Time=1463524668727, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:48,732 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524668494_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:48,733 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0] : 1
2016-05-17 22:37:48,733 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0]
2016-05-17 22:37:48,733 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:48,733 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668494_0001_1][Event:VERTEX_FINISHED]: vertexName=B1A8623957734E3BB908D418FD1932A0, vertexId=vertex_1463524668494_0001_1_00, initRequestedTime=1463524668675, initedTime=1463524668686, startRequestedTime=1463524668687, startedTime=1463524668687, finishTime=1463524668733, timeTaken=46, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524668724, Process_Duration=3, Process_End_Time=1463524668727, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524668696, firstTasksToStart=[ task_1463524668494_0001_1_00_000000 ], lastTaskFinishTime=1463524668730, lastTasksToFinish=[ task_1463524668494_0001_1_00_000000 ], minTaskDuration=34, maxTaskDuration=34, avgTaskDuration=34.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524668494_0001_1_00_000000 ], longestDurationTasks=[ task_1463524668494_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:48,734 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:48,734 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668494_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:48,734 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668494_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524668494_0001_00_000001, stoppedTime=1463524668734, exitStatus=0
2016-05-17 22:37:48,734 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668494_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:48,734 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:48,734 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:48,735 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524668494_0001_1
2016-05-17 22:37:48,735 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668494_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524668494_0001_1
2016-05-17 22:37:48,735 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 328C7037ED664F2E836047E1F9397E6B for vertex: vertex_1463524668494_0001_1_00 [B1A8623957734E3BB908D418FD1932A0]
2016-05-17 22:37:48,741 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524668494_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:48,741 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524668494_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:48,741 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524668494_0001_00_000001, Exiting
2016-05-17 22:37:48,742 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524668494_0001_00_000001
2016-05-17 22:37:48,742 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524668494_0001_00_000001
2016-05-17 22:37:48,744 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524668494_0001_1, queueSize=0
2016-05-17 22:37:48,744 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:48,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668494_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524668494_0001_1, startTime=1463524668675, finishTime=1463524668743, timeTaken=68, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524668724, Process_Duration=3, Process_End_Time=1463524668727, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:48,745 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524668494_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:48,745 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524668494_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:48,745 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524668494_0001_00_000001
2016-05-17 22:37:48,746 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524668494_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:48,746 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524668494_0001_00_000001 completed successfully
2016-05-17 22:37:48 Completed Dag: dag_1463524668494_0001_1
2016-05-17 22:37:48,750 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524668494_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:48,750 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524668494_0001_1_post
2016-05-17 22:37:48,750 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:48,750 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:48,750 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668494_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:37:48,751 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668494_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:37:48,752 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...LTS.ALL_'field2'_'field2', applicationId=application_1463524668494_0001
2016-05-17 22:37:48,754 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:48,754 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:48,754 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:48,755 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:48,755 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:48,756 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:48,756 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:48,756 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:48,756 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:48,757 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:48,757 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:48,757 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:48,757 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:48,764 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0150EE98CD6244A687E4271692B9C22B/D7706C2282BF4E008EF445DE57D6010C/.tez/application_1463524668494_0001
2016-05-17 22:37:48,765 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field2'/_temporary
2016-05-17 22:37:48,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.289
2016-05-17 22:37:48,792 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:48,792 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:48,833 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_'field2'
2016-05-17 22:37:48,855 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:48,855 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:48,884 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-17 22:37:48,891 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 22:37:48,892 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:48,892 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:48,893 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:48,906 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:48,907 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'
2016-05-17 22:37:48,916 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:48,916 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:48,916 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'"]
2016-05-17 22:37:48,916 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:48,917 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:48,917 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:48,918 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ULTS.ALL_'field2'_'field'
2016-05-17 22:37:48,918 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:48,934 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:48,938 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:48,939 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:48,939 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:48,948 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/32D54E471EBA4153828D1F6140F6F363/D764AAEBB5F041ED99676AD66C4C475F/.tez/application_1463524668939_0001 doesn't exist and is created
2016-05-17 22:37:48,964 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:48,964 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:48,965 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/32D54E471EBA4153828D1F6140F6F363/D764AAEBB5F041ED99676AD66C4C475F/.tez/application_1463524668939_0001
2016-05-17 22:37:48,967 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524668939_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:48,976 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:48,977 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:48,978 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:48,978 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:48,978 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/32D54E471EBA4153828D1F6140F6F363/D764AAEBB5F041ED99676AD66C4C475F/.tez/application_1463524668939_0001/localmode-log-dir
2016-05-17 22:37:48,979 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/32D54E471EBA4153828D1F6140F6F363/D764AAEBB5F041ED99676AD66C4C475F/.tez/application_1463524668939_0001/localmode-log-dir/history.txt.appattempt_1463524668939_0001_000000, maxErrors=10
2016-05-17 22:37:48,979 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:48,979 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524668939_0001_000000, appSubmitTime=1463524668967, launchTime=1463524668967
2016-05-17 22:37:48,981 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:48,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:48,985 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:48,985 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:48,987 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54609
2016-05-17 22:37:48,989 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:48,990 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524668939_0001_000000, startTime=1463524668990
2016-05-17 22:37:48,990 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:48,990 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:49,064 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:49,065 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:49,065 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_'field', applicationId=application_1463524668939_0001, dagName=[32D54E471EBA4153828D1F6140F6F363/D764AAEBB5F041ED99676AD66C4C475F] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-17 22:37:49,079 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [32D54E471EBA4153828D1F6140F6F363/D764AAEBB5F041ED99676AD66C4C475F] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-17 22:37:49,081 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524668939_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/32D54E471EBA4153828D1F6140F6F363/D764AAEBB5F041ED99676AD66C4C475F/.tez/application_1463524668939_0001/localmode-log-dir/dag_1463524668939_0001_1.dot
2016-05-17 22:37:49,085 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/32D54E471EBA4153828D1F6140F6F363/D764AAEBB5F041ED99676AD66C4C475F/.tez/application_1463524668939_0001/localmode-log-dir/dag_1463524668939_0001_1-tez-dag.pb.txt
2016-05-17 22:37:49,089 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524668939_0001_1
2016-05-17 22:37:49,089 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:49,089 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [32D54E471EBA4153828D1F6140F6F363/D764AAEBB5F041ED99676AD66C4C475F] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-17 22:37:49 Running Dag: dag_1463524668939_0001_1
2016-05-17 22:37:49,099 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:49,100 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668939_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524668939_0001_1, submitTime=1463524669079
2016-05-17 22:37:49,100 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DEB909746558490E852D4C4594754725
2016-05-17 22:37:49,101 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:49,101 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:49,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668939_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524668939_0001_1, initTime=1463524669100
2016-05-17 22:37:49,101 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524668939_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:49,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668939_0001_1][Event:DAG_STARTED]: dagID=dag_1463524668939_0001_1, startTime=1463524669101
2016-05-17 22:37:49,101 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:49,102 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:49,102 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_'field', applicationId=application_1463524668939_0001, dagName=[32D54E471EBA4153828D1F6140F6F363/D764AAEBB5F041ED99676AD66C4C475F] every.every/(1/1) ...ULTS.ALL_'field2'_'field'
2016-05-17 22:37:49,102 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524668939_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:49,102 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DEB909746558490E852D4C4594754725 : {0DDBCDCC482541BFB38068EF56CC7B96={InputName=0DDBCDCC482541BFB38068EF56CC7B96}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:49,102 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0DDBCDCC482541BFB38068EF56CC7B96, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:49,103 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725]
2016-05-17 22:37:49,103 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725]
2016-05-17 22:37:49,103 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725]
2016-05-17 22:37:49,103 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725]. Starting root input initializers: 1
2016-05-17 22:37:49,104 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:49,104 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0DDBCDCC482541BFB38068EF56CC7B96 on vertex vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725]
2016-05-17 22:37:49,105 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:49,107 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0DDBCDCC482541BFB38068EF56CC7B96 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:49,107 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:49,108 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:49,109 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:49,110 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0DDBCDCC482541BFB38068EF56CC7B96 on vertex vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725]
2016-05-17 22:37:49,110 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:49,110 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0DDBCDCC482541BFB38068EF56CC7B96=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:49,111 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725] parallelism set to 1
2016-05-17 22:37:49,111 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:49,111 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668939_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524668939_0001_1_00, eventCount=1
2016-05-17 22:37:49,111 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725]
2016-05-17 22:37:49,111 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=5E0666F37B3049F6981FD0C70F17CF9D, vertexId=vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:49,111 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=5E0666F37B3049F6981FD0C70F17CF9D, vertexId=vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725]
2016-05-17 22:37:49,113 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=5E0666F37B3049F6981FD0C70F17CF9D, vertexName=DEB909746558490E852D4C4594754725, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:49,113 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=5E0666F37B3049F6981FD0C70F17CF9D, vertexId=vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725]
2016-05-17 22:37:49,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668939_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DEB909746558490E852D4C4594754725, vertexId=vertex_1463524668939_0001_1_00, initRequestedTime=1463524669102, initedTime=1463524669115, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:49,115 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725] with distanceFromRoot: 0
2016-05-17 22:37:49,115 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:49,115 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:49,116 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DEB909746558490E852D4C4594754725
2016-05-17 22:37:49,116 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725]
2016-05-17 22:37:49,116 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668939_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524668939_0001_1_00, startRequestedTime=1463524669116, startedTime=1463524669116
2016-05-17 22:37:49,116 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:49,116 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:49,117 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:49,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668939_0001_1][Event:TASK_STARTED]: vertexName=DEB909746558490E852D4C4594754725, taskId=task_1463524668939_0001_1_00_000000, scheduledTime=1463524669117, launchTime=1463524669117
2016-05-17 22:37:49,117 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524668939_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:49,117 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725]
2016-05-17 22:37:49,118 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [32D54E471EBA4153828D1F6140F6F363/D764AAEBB5F041ED99676AD66C4C475F] every.every/(1/1) ...ULTS.ALL_'field2'_'field', VertexName: DEB909746558490E852D4C4594754725, VertexParallelism: 1, TaskAttemptID:attempt_1463524668939_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0DDBCDCC482541BFB38068EF56CC7B96, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5E0666F37B3049F6981FD0C70F17CF9D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:49,118 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524668939_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:49,118 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:49,119 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524668939_0001, with dag id: dag_1463524668939_0001_1
2016-05-17 22:37:49,119 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:49,120 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:49,120 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:49,120 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524668939_0001_00_000001
2016-05-17 22:37:49,121 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668939_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:49,122 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524668939_0001_00_000001
2016-05-17 22:37:49,122 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:49,123 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668939_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:49,123 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:49,123 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668939_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524668939_0001_00_000001, launchTime=1463524669123
2016-05-17 22:37:49,124 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524668939_0001_00_000001
2016-05-17 22:37:49,124 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524668939_0001_1_00_000000_0] to container: [container_1463524668939_0001_00_000001]
2016-05-17 22:37:49,125 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668939_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:49,125 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524668939_0001_00_000001 given task: attempt_1463524668939_0001_1_00_000000_0
2016-05-17 22:37:49,125 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524668939_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524668939_0001_1_00_000000_0
2016-05-17 22:37:49,125 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524668939_0001_1_00_000000_0] started. Is using containerId: [container_1463524668939_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:49,126 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668939_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DEB909746558490E852D4C4594754725, taskAttemptId=attempt_1463524668939_0001_1_00_000000_0, startTime=1463524669125, containerId=container_1463524668939_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524668939_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:49,126 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524668939_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:49,126 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524668939_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:49,126 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:49 Starting to run new task attempt: attempt_1463524668939_0001_1_00_000000_0
2016-05-17 22:37:49,126 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524668939_0001_1_00_000000_0
2016-05-17 22:37:49,126 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:49,127 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:49,127 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:49,127 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:49,127 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:49,128 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:49,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [32D54E471EBA4153828D1F6140F6F363/D764AAEBB5F041ED99676AD66C4C475F] every.every/(1/1) ...ULTS.ALL_'field2'_'field', VertexName: DEB909746558490E852D4C4594754725, VertexParallelism: 1, TaskAttemptID:attempt_1463524668939_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0DDBCDCC482541BFB38068EF56CC7B96, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=5E0666F37B3049F6981FD0C70F17CF9D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:49,128 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:49,128 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524668939_0001_1_00_000000_0
2016-05-17 22:37:49,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:49,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:49,129 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524668939_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:49,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0DDBCDCC482541BFB38068EF56CC7B96, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:49,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:49,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:49,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0DDBCDCC482541BFB38068EF56CC7B96
2016-05-17 22:37:49,130 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0DDBCDCC482541BFB38068EF56CC7B96
2016-05-17 22:37:49,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=5E0666F37B3049F6981FD0C70F17CF9D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:49,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:49,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 5E0666F37B3049F6981FD0C70F17CF9D
2016-05-17 22:37:49,132 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:49,132 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:49,132 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0DDBCDCC482541BFB38068EF56CC7B96
2016-05-17 22:37:49,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0DDBCDCC482541BFB38068EF56CC7B96
2016-05-17 22:37:49,133 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:49,132 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 5E0666F37B3049F6981FD0C70F17CF9D
2016-05-17 22:37:49,140 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DEB909746558490E852D4C4594754725, ordinal: 0
2016-05-17 22:37:49,140 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DEB909746558490E852D4C4594754725, mem on start (mb), free: 195, total: 359, max: 672
2016-05-17 22:37:49,140 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:49,140 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:49,140 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:49,144 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:49,145 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 5E0666F37B3049F6981FD0C70F17CF9D
2016-05-17 22:37:49,145 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:49,145 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:49,145 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:49,145 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:49,145 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:49,145 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:49,145 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:49,146 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:49,146 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0DDBCDCC482541BFB38068EF56CC7B96, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:49,146 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 5E0666F37B3049F6981FD0C70F17CF9D, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:49,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:49,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0DDBCDCC482541BFB38068EF56CC7B96
2016-05-17 22:37:49,146 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:49,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0DDBCDCC482541BFB38068EF56CC7B96
2016-05-17 22:37:49,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0DDBCDCC482541BFB38068EF56CC7B96 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:49,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:49,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:49,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:49,149 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524668939_0001_1_00_000000_0
2016-05-17 22:37:49,149 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:49,149 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6124022e
2016-05-17 22:37:49,150 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6124022e
2016-05-17 22:37:49,150 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:49,150 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:49,153 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:49,153 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0DDBCDCC482541BFB38068EF56CC7B96
2016-05-17 22:37:49,153 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'"], id: 5E0666F37B3049F6981FD0C70F17CF9D
2016-05-17 22:37:49,154 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'"]
2016-05-17 22:37:49,154 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:49,154 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:49,154 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DEB909746558490E852D4C4594754725, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:49,157 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:49,157 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:49,158 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524668939_0001_1_00_000000_0
2016-05-17 22:37:49,158 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524668939_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:49,159 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524668939_0001_1_00_000000_0
2016-05-17 22:37:49,159 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DEB909746558490E852D4C4594754725, mem on close (mb), free: 194, total: 359, max: 672
2016-05-17 22:37:49 Completed running task attempt: attempt_1463524668939_0001_1_00_000000_0
2016-05-17 22:37:49,160 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524668939_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:49,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524669155, Process_Duration=4, Process_End_Time=1463524669159, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:49,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668939_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DEB909746558490E852D4C4594754725, taskAttemptId=attempt_1463524668939_0001_1_00_000000_0, startTime=1463524669125, finishTime=1463524669161, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524669155, Process_Duration=4, Process_End_Time=1463524669159, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:49,161 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:49,161 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:49,161 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:49,162 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524668939_0001_00_000001
2016-05-17 22:37:49,161 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524668939_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:49,162 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524668939_0001_1_00_000000_0
2016-05-17 22:37:49,163 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:49,163 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:49,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668939_0001_1][Event:TASK_FINISHED]: vertexName=DEB909746558490E852D4C4594754725, taskId=task_1463524668939_0001_1_00_000000, startTime=1463524669125, finishTime=1463524669163, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1463524668939_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524669155, Process_Duration=4, Process_End_Time=1463524669159, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:49,163 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524668939_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:49,164 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725] : 1
2016-05-17 22:37:49,164 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725]
2016-05-17 22:37:49,164 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:49,164 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668939_0001_1][Event:VERTEX_FINISHED]: vertexName=DEB909746558490E852D4C4594754725, vertexId=vertex_1463524668939_0001_1_00, initRequestedTime=1463524669102, initedTime=1463524669115, startRequestedTime=1463524669116, startedTime=1463524669116, finishTime=1463524669164, timeTaken=48, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524669155, Process_Duration=4, Process_End_Time=1463524669159, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524669125, firstTasksToStart=[ task_1463524668939_0001_1_00_000000 ], lastTaskFinishTime=1463524669161, lastTasksToFinish=[ task_1463524668939_0001_1_00_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524668939_0001_1_00_000000 ], longestDurationTasks=[ task_1463524668939_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:49,165 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:49,165 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668939_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:49,165 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668939_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524668939_0001_00_000001, stoppedTime=1463524669165, exitStatus=0
2016-05-17 22:37:49,165 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:49,165 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668939_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:49,165 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:49,165 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524668939_0001_1
2016-05-17 22:37:49,165 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668939_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524668939_0001_1
2016-05-17 22:37:49,166 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 5E0666F37B3049F6981FD0C70F17CF9D for vertex: vertex_1463524668939_0001_1_00 [DEB909746558490E852D4C4594754725]
2016-05-17 22:37:49,173 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524668939_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:49,173 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524668939_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:49,173 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524668939_0001_00_000001, Exiting
2016-05-17 22:37:49,173 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524668939_0001_00_000001
2016-05-17 22:37:49,173 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524668939_0001_00_000001
2016-05-17 22:37:49,173 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524668939_0001_1, queueSize=0
2016-05-17 22:37:49,174 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524668939_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:49,174 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:49,174 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524668939_0001_00_000001 completed successfully
2016-05-17 22:37:49,174 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524668939_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524668939_0001_1, startTime=1463524669101, finishTime=1463524669172, timeTaken=71, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524669155, Process_Duration=4, Process_End_Time=1463524669159, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:49,174 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524668939_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:49,175 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524668939_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:49,175 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524668939_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:49,175 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524668939_0001_00_000001
2016-05-17 22:37:49 Completed Dag: dag_1463524668939_0001_1
2016-05-17 22:37:49,176 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524668939_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:49,176 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524668939_0001_1_post
2016-05-17 22:37:49,177 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:49,177 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:49,181 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ULTS.ALL_'field2'_'field', applicationId=application_1463524668939_0001
2016-05-17 22:37:49,182 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:49,183 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:49,183 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:49,183 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:49,183 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:49,183 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:49,183 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:49,183 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:49,184 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:49,184 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:49,184 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:49,184 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:49,184 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:49,188 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/32D54E471EBA4153828D1F6140F6F363/D764AAEBB5F041ED99676AD66C4C475F/.tez/application_1463524668939_0001
2016-05-17 22:37:49,189 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_'field'/_temporary
2016-05-17 22:37:49,189 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.273
2016-05-17 22:37:49,255 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_'field'
2016-05-17 22:37:49,276 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:49,276 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:49,280 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:49,280 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:49,305 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 22:37:49,313 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 22:37:49,314 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:49,314 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:49,314 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:49,329 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:49,329 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS
2016-05-17 22:37:49,337 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:49,338 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:49,338 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS"]
2016-05-17 22:37:49,338 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:49,338 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:49,338 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:49,339 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-17 22:37:49,339 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:49,352 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:49,356 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:49,357 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:49,357 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:49,366 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0F2844CB40E34D29B72CB73437EE7C9F/E20F4B781C0B4C0580997635FD08780B/.tez/application_1463524669357_0001 doesn't exist and is created
2016-05-17 22:37:49,384 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:49,384 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:49,384 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0F2844CB40E34D29B72CB73437EE7C9F/E20F4B781C0B4C0580997635FD08780B/.tez/application_1463524669357_0001
2016-05-17 22:37:49,388 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524669357_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:49,398 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:49,399 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:49,400 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:49,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:49,401 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0F2844CB40E34D29B72CB73437EE7C9F/E20F4B781C0B4C0580997635FD08780B/.tez/application_1463524669357_0001/localmode-log-dir
2016-05-17 22:37:49,401 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0F2844CB40E34D29B72CB73437EE7C9F/E20F4B781C0B4C0580997635FD08780B/.tez/application_1463524669357_0001/localmode-log-dir/history.txt.appattempt_1463524669357_0001_000000, maxErrors=10
2016-05-17 22:37:49,402 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:49,402 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524669357_0001_000000, appSubmitTime=1463524669387, launchTime=1463524669387
2016-05-17 22:37:49,409 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:49,409 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:38598
2016-05-17 22:37:49,410 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:49,411 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:49,411 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:49,412 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:49,414 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524669357_0001_000000, startTime=1463524669414
2016-05-17 22:37:49,414 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:49,414 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:49,484 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:49,484 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:49,484 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_RESULTS, applicationId=application_1463524669357_0001, dagName=[0F2844CB40E34D29B72CB73437EE7C9F/E20F4B781C0B4C0580997635FD08780B] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-17 22:37:49,499 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [0F2844CB40E34D29B72CB73437EE7C9F/E20F4B781C0B4C0580997635FD08780B] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-17 22:37:49,500 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524669357_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0F2844CB40E34D29B72CB73437EE7C9F/E20F4B781C0B4C0580997635FD08780B/.tez/application_1463524669357_0001/localmode-log-dir/dag_1463524669357_0001_1.dot
2016-05-17 22:37:49,501 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0F2844CB40E34D29B72CB73437EE7C9F/E20F4B781C0B4C0580997635FD08780B/.tez/application_1463524669357_0001/localmode-log-dir/dag_1463524669357_0001_1-tez-dag.pb.txt
2016-05-17 22:37:49,505 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524669357_0001_1
2016-05-17 22:37:49,505 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:49,505 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [0F2844CB40E34D29B72CB73437EE7C9F/E20F4B781C0B4C0580997635FD08780B] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-17 22:37:49 Running Dag: dag_1463524669357_0001_1
2016-05-17 22:37:49,518 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:49,518 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669357_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524669357_0001_1, submitTime=1463524669499
2016-05-17 22:37:49,518 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex AFA55AB63DAD418C842CD8CDDDF2C547
2016-05-17 22:37:49,518 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:49,519 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:49,519 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669357_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524669357_0001_1, initTime=1463524669518
2016-05-17 22:37:49,519 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524669357_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:49,519 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669357_0001_1][Event:DAG_STARTED]: dagID=dag_1463524669357_0001_1, startTime=1463524669519
2016-05-17 22:37:49,519 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:49,519 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524669357_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:49,520 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AFA55AB63DAD418C842CD8CDDDF2C547 : {2FA2731B8434483CA87FE1A6AE5BC064={InputName=2FA2731B8434483CA87FE1A6AE5BC064}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:49,520 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2FA2731B8434483CA87FE1A6AE5BC064, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:49,520 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547]
2016-05-17 22:37:49,520 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547]
2016-05-17 22:37:49,520 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547]
2016-05-17 22:37:49,520 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547]. Starting root input initializers: 1
2016-05-17 22:37:49,520 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:49,520 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:49,521 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2FA2731B8434483CA87FE1A6AE5BC064 on vertex vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547]
2016-05-17 22:37:49,521 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ULTS.ALL_'field2'_RESULTS, applicationId=application_1463524669357_0001, dagName=[0F2844CB40E34D29B72CB73437EE7C9F/E20F4B781C0B4C0580997635FD08780B] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS
2016-05-17 22:37:49,522 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:49,523 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2FA2731B8434483CA87FE1A6AE5BC064 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:49,523 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:49,523 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:49,525 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:49,525 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2FA2731B8434483CA87FE1A6AE5BC064 on vertex vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547]
2016-05-17 22:37:49,526 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:49,526 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2FA2731B8434483CA87FE1A6AE5BC064=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:49,526 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547] parallelism set to 1
2016-05-17 22:37:49,526 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:49,526 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669357_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524669357_0001_1_00, eventCount=1
2016-05-17 22:37:49,526 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547]
2016-05-17 22:37:49,527 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B07C29F1E98C46BF82D8A436C456C07F, vertexId=vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:49,527 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B07C29F1E98C46BF82D8A436C456C07F, vertexId=vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547]
2016-05-17 22:37:49,528 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B07C29F1E98C46BF82D8A436C456C07F, vertexName=AFA55AB63DAD418C842CD8CDDDF2C547, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:49,529 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B07C29F1E98C46BF82D8A436C456C07F, vertexId=vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547]
2016-05-17 22:37:49,529 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:49,529 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669357_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AFA55AB63DAD418C842CD8CDDDF2C547, vertexId=vertex_1463524669357_0001_1_00, initRequestedTime=1463524669520, initedTime=1463524669529, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:49,530 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547] with distanceFromRoot: 0
2016-05-17 22:37:49,530 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:49,530 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AFA55AB63DAD418C842CD8CDDDF2C547
2016-05-17 22:37:49,530 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547]
2016-05-17 22:37:49,530 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:49,531 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669357_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524669357_0001_1_00, startRequestedTime=1463524669530, startedTime=1463524669530
2016-05-17 22:37:49,531 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:49,531 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:49,531 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669357_0001_1][Event:TASK_STARTED]: vertexName=AFA55AB63DAD418C842CD8CDDDF2C547, taskId=task_1463524669357_0001_1_00_000000, scheduledTime=1463524669531, launchTime=1463524669531
2016-05-17 22:37:49,532 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524669357_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:49,532 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547]
2016-05-17 22:37:49,532 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0F2844CB40E34D29B72CB73437EE7C9F/E20F4B781C0B4C0580997635FD08780B] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS, VertexName: AFA55AB63DAD418C842CD8CDDDF2C547, VertexParallelism: 1, TaskAttemptID:attempt_1463524669357_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2FA2731B8434483CA87FE1A6AE5BC064, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B07C29F1E98C46BF82D8A436C456C07F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:49,533 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524669357_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:49,533 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:49,533 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:49,534 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:49,534 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:49,534 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524669357_0001_00_000001
2016-05-17 22:37:49,534 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524669357_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:49,536 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524669357_0001_00_000001
2016-05-17 22:37:49,536 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:49,537 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524669357_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:49,537 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:49,537 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669357_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524669357_0001_00_000001, launchTime=1463524669537
2016-05-17 22:37:49,538 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524669357_0001_00_000001
2016-05-17 22:37:49,538 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524669357_0001_1_00_000000_0] to container: [container_1463524669357_0001_00_000001]
2016-05-17 22:37:49,539 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524669357_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:49,539 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524669357_0001_00_000001 given task: attempt_1463524669357_0001_1_00_000000_0
2016-05-17 22:37:49,539 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524669357_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524669357_0001_1_00_000000_0
2016-05-17 22:37:49,539 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524669357_0001_1_00_000000_0] started. Is using containerId: [container_1463524669357_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:49,539 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669357_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AFA55AB63DAD418C842CD8CDDDF2C547, taskAttemptId=attempt_1463524669357_0001_1_00_000000_0, startTime=1463524669539, containerId=container_1463524669357_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524669357_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:49,539 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524669357_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:49 Starting to run new task attempt: attempt_1463524669357_0001_1_00_000000_0
2016-05-17 22:37:49,540 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524669357_0001_1_00_000000_0
2016-05-17 22:37:49,540 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524669357_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:49,540 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:49,540 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:49,540 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:49,540 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:49,540 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:49,540 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:49,541 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:49,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0F2844CB40E34D29B72CB73437EE7C9F/E20F4B781C0B4C0580997635FD08780B] every.every/(1/1) ...ULTS.ALL_'field2'_RESULTS, VertexName: AFA55AB63DAD418C842CD8CDDDF2C547, VertexParallelism: 1, TaskAttemptID:attempt_1463524669357_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2FA2731B8434483CA87FE1A6AE5BC064, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B07C29F1E98C46BF82D8A436C456C07F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:49,541 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:49,541 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524669357_0001_1_00_000000_0
2016-05-17 22:37:49,541 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524669357_0001, with dag id: dag_1463524669357_0001_1
2016-05-17 22:37:49,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:49,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:49,543 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:49,543 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B07C29F1E98C46BF82D8A436C456C07F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:49,543 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:49,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B07C29F1E98C46BF82D8A436C456C07F
2016-05-17 22:37:49,544 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:49,544 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B07C29F1E98C46BF82D8A436C456C07F
2016-05-17 22:37:49,545 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:49,548 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524669357_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:49,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2FA2731B8434483CA87FE1A6AE5BC064, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:49,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:49,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2FA2731B8434483CA87FE1A6AE5BC064
2016-05-17 22:37:49,550 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2FA2731B8434483CA87FE1A6AE5BC064
2016-05-17 22:37:49,553 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:49,554 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2FA2731B8434483CA87FE1A6AE5BC064
2016-05-17 22:37:49,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2FA2731B8434483CA87FE1A6AE5BC064
2016-05-17 22:37:49,558 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:49,558 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B07C29F1E98C46BF82D8A436C456C07F
2016-05-17 22:37:49,559 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AFA55AB63DAD418C842CD8CDDDF2C547, ordinal: 0
2016-05-17 22:37:49,559 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AFA55AB63DAD418C842CD8CDDDF2C547, mem on start (mb), free: 114, total: 359, max: 672
2016-05-17 22:37:49,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:49,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:49,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:49,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:49,560 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:49,560 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:49,561 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:49,561 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:49,561 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:49,561 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:49,561 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:49,562 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B07C29F1E98C46BF82D8A436C456C07F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:49,562 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2FA2731B8434483CA87FE1A6AE5BC064, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:49,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:49,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2FA2731B8434483CA87FE1A6AE5BC064
2016-05-17 22:37:49,562 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:49,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2FA2731B8434483CA87FE1A6AE5BC064
2016-05-17 22:37:49,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2FA2731B8434483CA87FE1A6AE5BC064 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:49,563 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:49,563 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:49,563 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:49,564 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524669357_0001_1_00_000000_0
2016-05-17 22:37:49,564 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:49,566 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@38fa1a94
2016-05-17 22:37:49,566 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:49,566 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@38fa1a94
2016-05-17 22:37:49,580 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:49,580 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:49,581 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2FA2731B8434483CA87FE1A6AE5BC064
2016-05-17 22:37:49,581 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS"], id: B07C29F1E98C46BF82D8A436C456C07F
2016-05-17 22:37:49,582 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS"]
2016-05-17 22:37:49,583 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:49,583 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:49,583 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AFA55AB63DAD418C842CD8CDDDF2C547, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:49,586 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:49,586 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:49,586 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524669357_0001_1_00_000000_0
2016-05-17 22:37:49,587 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524669357_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:49,587 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524669357_0001_1_00_000000_0
2016-05-17 22:37:49,587 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AFA55AB63DAD418C842CD8CDDDF2C547, mem on close (mb), free: 298, total: 353, max: 672
2016-05-17 22:37:49 Completed running task attempt: attempt_1463524669357_0001_1_00_000000_0
2016-05-17 22:37:49,589 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524669357_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:49,590 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669357_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AFA55AB63DAD418C842CD8CDDDF2C547, taskAttemptId=attempt_1463524669357_0001_1_00_000000_0, startTime=1463524669539, finishTime=1463524669590, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370671616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524669583, Process_Duration=4, Process_End_Time=1463524669587, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:49,590 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524669357_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:49,590 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524669357_0001_1_00_000000_0
2016-05-17 22:37:49,590 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:49,590 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:49,590 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669357_0001_1][Event:TASK_FINISHED]: vertexName=AFA55AB63DAD418C842CD8CDDDF2C547, taskId=task_1463524669357_0001_1_00_000000, startTime=1463524669539, finishTime=1463524669590, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1463524669357_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370671616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524669583, Process_Duration=4, Process_End_Time=1463524669587, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:49,591 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524669357_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:49,591 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:49,591 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547] : 1
2016-05-17 22:37:49,591 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547]
2016-05-17 22:37:49,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370671616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524669583, Process_Duration=4, Process_End_Time=1463524669587, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:49,591 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:49,592 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:49,592 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:49,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669357_0001_1][Event:VERTEX_FINISHED]: vertexName=AFA55AB63DAD418C842CD8CDDDF2C547, vertexId=vertex_1463524669357_0001_1_00, initRequestedTime=1463524669520, initedTime=1463524669529, startRequestedTime=1463524669530, startedTime=1463524669530, finishTime=1463524669591, timeTaken=61, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370671616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524669583, Process_Duration=4, Process_End_Time=1463524669587, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524669539, firstTasksToStart=[ task_1463524669357_0001_1_00_000000 ], lastTaskFinishTime=1463524669590, lastTasksToFinish=[ task_1463524669357_0001_1_00_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524669357_0001_1_00_000000 ], longestDurationTasks=[ task_1463524669357_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:49,592 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:49,592 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524669357_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:49,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669357_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524669357_0001_00_000001, stoppedTime=1463524669592, exitStatus=0
2016-05-17 22:37:49,593 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524669357_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:49,593 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:49,593 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524669357_0001_1
2016-05-17 22:37:49,592 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524669357_0001_00_000001
2016-05-17 22:37:49,593 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:49,594 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524669357_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:49,594 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524669357_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:49,594 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669357_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524669357_0001_1
2016-05-17 22:37:49,594 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524669357_0001_00_000001, Exiting
2016-05-17 22:37:49,594 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524669357_0001_00_000001
2016-05-17 22:37:49,595 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524669357_0001_00_000001
2016-05-17 22:37:49,595 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: B07C29F1E98C46BF82D8A436C456C07F for vertex: vertex_1463524669357_0001_1_00 [AFA55AB63DAD418C842CD8CDDDF2C547]
2016-05-17 22:37:49,596 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524669357_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:49,597 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524669357_0001_00_000001 completed successfully
2016-05-17 22:37:49,604 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524669357_0001_1, queueSize=0
2016-05-17 22:37:49,604 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:49,604 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669357_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524669357_0001_1, startTime=1463524669519, finishTime=1463524669603, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370671616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524669583, Process_Duration=4, Process_End_Time=1463524669587, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:49,605 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524669357_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:49,605 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524669357_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:49,605 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524669357_0001_00_000001
2016-05-17 22:37:49,606 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524669357_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:49 Completed Dag: dag_1463524669357_0001_1
2016-05-17 22:37:49,607 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524669357_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:49,607 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524669357_0001_1_post
2016-05-17 22:37:49,607 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:49,608 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:49,611 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ULTS.ALL_'field2'_RESULTS, applicationId=application_1463524669357_0001
2016-05-17 22:37:49,613 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:49,613 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:49,613 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:49,613 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:49,613 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:49,614 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:49,614 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:49,614 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:49,614 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:49,615 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:49,615 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:49,615 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:49,615 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:49,619 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/0F2844CB40E34D29B72CB73437EE7C9F/E20F4B781C0B4C0580997635FD08780B/.tez/application_1463524669357_0001
2016-05-17 22:37:49,620 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_RESULTS/_temporary
2016-05-17 22:37:49,620 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.282
2016-05-17 22:37:49,688 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_RESULTS
2016-05-17 22:37:49,709 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:49,709 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:49,731 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:49,731 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:49,737 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 22:37:49,743 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 22:37:49,743 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:49,743 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:49,744 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:49,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:49,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL
2016-05-17 22:37:49,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:49,766 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:49,766 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL"]
2016-05-17 22:37:49,766 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:49,766 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:49,767 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:49,767 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-17 22:37:49,767 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:49,779 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:49,783 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:49,784 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:49,784 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:49,792 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ACB66502C4A04AFAAA740BA3F8A12758/B6DE6481DD224796BE2D7C04EDB91E01/.tez/application_1463524669784_0001 doesn't exist and is created
2016-05-17 22:37:49,809 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:49,809 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:49,809 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ACB66502C4A04AFAAA740BA3F8A12758/B6DE6481DD224796BE2D7C04EDB91E01/.tez/application_1463524669784_0001
2016-05-17 22:37:49,813 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524669784_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:49,822 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:49,823 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:49,824 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:49,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:49,824 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ACB66502C4A04AFAAA740BA3F8A12758/B6DE6481DD224796BE2D7C04EDB91E01/.tez/application_1463524669784_0001/localmode-log-dir
2016-05-17 22:37:49,825 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ACB66502C4A04AFAAA740BA3F8A12758/B6DE6481DD224796BE2D7C04EDB91E01/.tez/application_1463524669784_0001/localmode-log-dir/history.txt.appattempt_1463524669784_0001_000000, maxErrors=10
2016-05-17 22:37:49,825 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:49,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524669784_0001_000000, appSubmitTime=1463524669812, launchTime=1463524669813
2016-05-17 22:37:49,829 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:49,832 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:49,832 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:59274
2016-05-17 22:37:49,836 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:49,837 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:49,837 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:49,839 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524669784_0001_000000, startTime=1463524669838
2016-05-17 22:37:49,839 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:49,839 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:49,909 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:49,909 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:49,909 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ..._RESULTS.ALL_'field2'_ALL, applicationId=application_1463524669784_0001, dagName=[ACB66502C4A04AFAAA740BA3F8A12758/B6DE6481DD224796BE2D7C04EDB91E01] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-17 22:37:49,923 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [ACB66502C4A04AFAAA740BA3F8A12758/B6DE6481DD224796BE2D7C04EDB91E01] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-17 22:37:49,924 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524669784_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ACB66502C4A04AFAAA740BA3F8A12758/B6DE6481DD224796BE2D7C04EDB91E01/.tez/application_1463524669784_0001/localmode-log-dir/dag_1463524669784_0001_1.dot
2016-05-17 22:37:49,925 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ACB66502C4A04AFAAA740BA3F8A12758/B6DE6481DD224796BE2D7C04EDB91E01/.tez/application_1463524669784_0001/localmode-log-dir/dag_1463524669784_0001_1-tez-dag.pb.txt
2016-05-17 22:37:49,929 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524669784_0001_1
2016-05-17 22:37:49,929 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:49,929 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [ACB66502C4A04AFAAA740BA3F8A12758/B6DE6481DD224796BE2D7C04EDB91E01] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-17 22:37:49 Running Dag: dag_1463524669784_0001_1
2016-05-17 22:37:49,941 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:49,941 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669784_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524669784_0001_1, submitTime=1463524669923
2016-05-17 22:37:49,942 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EFC74D2D40B94F58B6ABB49FE5D551AD
2016-05-17 22:37:49,942 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:49,942 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:49,942 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669784_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524669784_0001_1, initTime=1463524669941
2016-05-17 22:37:49,942 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524669784_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:49,943 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669784_0001_1][Event:DAG_STARTED]: dagID=dag_1463524669784_0001_1, startTime=1463524669942
2016-05-17 22:37:49,943 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:49,943 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:49,943 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ..._RESULTS.ALL_'field2'_ALL, applicationId=application_1463524669784_0001, dagName=[ACB66502C4A04AFAAA740BA3F8A12758/B6DE6481DD224796BE2D7C04EDB91E01] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL
2016-05-17 22:37:49,943 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524669784_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:49,943 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EFC74D2D40B94F58B6ABB49FE5D551AD : {E5EAE7FADD5C4C2DB7889C3A1C2CAC37={InputName=E5EAE7FADD5C4C2DB7889C3A1C2CAC37}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:49,943 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E5EAE7FADD5C4C2DB7889C3A1C2CAC37, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:49,943 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD]
2016-05-17 22:37:49,944 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD]
2016-05-17 22:37:49,944 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD]
2016-05-17 22:37:49,944 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD]. Starting root input initializers: 1
2016-05-17 22:37:49,944 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:49,945 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E5EAE7FADD5C4C2DB7889C3A1C2CAC37 on vertex vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD]
2016-05-17 22:37:49,946 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:49,947 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E5EAE7FADD5C4C2DB7889C3A1C2CAC37 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:49,947 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:49,947 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:49,948 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:49,949 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E5EAE7FADD5C4C2DB7889C3A1C2CAC37 on vertex vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD]
2016-05-17 22:37:49,949 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:49,949 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E5EAE7FADD5C4C2DB7889C3A1C2CAC37=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:49,950 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD] parallelism set to 1
2016-05-17 22:37:49,950 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:49,950 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669784_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524669784_0001_1_00, eventCount=1
2016-05-17 22:37:49,950 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD]
2016-05-17 22:37:49,950 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1CC56B9B024F4B1DB81D08A7C0782D7F, vertexId=vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:49,950 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1CC56B9B024F4B1DB81D08A7C0782D7F, vertexId=vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD]
2016-05-17 22:37:49,952 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1CC56B9B024F4B1DB81D08A7C0782D7F, vertexName=EFC74D2D40B94F58B6ABB49FE5D551AD, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:49,953 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1CC56B9B024F4B1DB81D08A7C0782D7F, vertexId=vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD]
2016-05-17 22:37:49,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669784_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EFC74D2D40B94F58B6ABB49FE5D551AD, vertexId=vertex_1463524669784_0001_1_00, initRequestedTime=1463524669943, initedTime=1463524669954, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:49,954 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD] with distanceFromRoot: 0
2016-05-17 22:37:49,954 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:49,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:49,954 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EFC74D2D40B94F58B6ABB49FE5D551AD
2016-05-17 22:37:49,954 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD]
2016-05-17 22:37:49,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:49,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669784_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524669784_0001_1_00, startRequestedTime=1463524669954, startedTime=1463524669954
2016-05-17 22:37:49,955 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:49,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:49,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669784_0001_1][Event:TASK_STARTED]: vertexName=EFC74D2D40B94F58B6ABB49FE5D551AD, taskId=task_1463524669784_0001_1_00_000000, scheduledTime=1463524669955, launchTime=1463524669955
2016-05-17 22:37:49,956 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524669784_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:49,956 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD]
2016-05-17 22:37:49,956 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [ACB66502C4A04AFAAA740BA3F8A12758/B6DE6481DD224796BE2D7C04EDB91E01] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL, VertexName: EFC74D2D40B94F58B6ABB49FE5D551AD, VertexParallelism: 1, TaskAttemptID:attempt_1463524669784_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E5EAE7FADD5C4C2DB7889C3A1C2CAC37, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1CC56B9B024F4B1DB81D08A7C0782D7F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:49,957 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524669784_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:49,957 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:49,958 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:49,958 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:49,958 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:49,959 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524669784_0001_00_000001
2016-05-17 22:37:49,959 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524669784_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:49,961 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524669784_0001_00_000001
2016-05-17 22:37:49,961 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:49,961 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524669784_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:49,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669784_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524669784_0001_00_000001, launchTime=1463524669961
2016-05-17 22:37:49,961 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:49,961 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524669784_0001, with dag id: dag_1463524669784_0001_1
2016-05-17 22:37:49,965 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524669784_0001_00_000001
2016-05-17 22:37:49,966 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524669784_0001_1_00_000000_0] to container: [container_1463524669784_0001_00_000001]
2016-05-17 22:37:49,966 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524669784_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:49,966 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524669784_0001_00_000001 given task: attempt_1463524669784_0001_1_00_000000_0
2016-05-17 22:37:49,967 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524669784_0001_00_000001: 1 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524669784_0001_1_00_000000_0
2016-05-17 22:37:49 Starting to run new task attempt: attempt_1463524669784_0001_1_00_000000_0
2016-05-17 22:37:49,967 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524669784_0001_1_00_000000_0
2016-05-17 22:37:49,967 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:49,967 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:49,967 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:49,967 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:49,967 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:49,967 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:49,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [ACB66502C4A04AFAAA740BA3F8A12758/B6DE6481DD224796BE2D7C04EDB91E01] every.every/(1/1) ..._RESULTS.ALL_'field2'_ALL, VertexName: EFC74D2D40B94F58B6ABB49FE5D551AD, VertexParallelism: 1, TaskAttemptID:attempt_1463524669784_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E5EAE7FADD5C4C2DB7889C3A1C2CAC37, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=1CC56B9B024F4B1DB81D08A7C0782D7F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:49,968 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:49,968 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524669784_0001_1_00_000000_0
2016-05-17 22:37:49,968 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524669784_0001_1_00_000000_0] started. Is using containerId: [container_1463524669784_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:49,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:49,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:49,968 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:49,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669784_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EFC74D2D40B94F58B6ABB49FE5D551AD, taskAttemptId=attempt_1463524669784_0001_1_00_000000_0, startTime=1463524669967, containerId=container_1463524669784_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524669784_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:49,969 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524669784_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:49,969 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524669784_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:49,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:49,970 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524669784_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:49,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1CC56B9B024F4B1DB81D08A7C0782D7F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:49,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:49,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1CC56B9B024F4B1DB81D08A7C0782D7F
2016-05-17 22:37:49,971 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:49,971 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1CC56B9B024F4B1DB81D08A7C0782D7F
2016-05-17 22:37:49,971 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E5EAE7FADD5C4C2DB7889C3A1C2CAC37, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:49,972 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:49,972 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E5EAE7FADD5C4C2DB7889C3A1C2CAC37
2016-05-17 22:37:49,972 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E5EAE7FADD5C4C2DB7889C3A1C2CAC37
2016-05-17 22:37:49,972 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:49,979 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:49,979 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E5EAE7FADD5C4C2DB7889C3A1C2CAC37
2016-05-17 22:37:49,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E5EAE7FADD5C4C2DB7889C3A1C2CAC37
2016-05-17 22:37:49,985 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EFC74D2D40B94F58B6ABB49FE5D551AD, ordinal: 0
2016-05-17 22:37:49,985 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:49,985 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EFC74D2D40B94F58B6ABB49FE5D551AD, mem on start (mb), free: 221, total: 353, max: 672
2016-05-17 22:37:49,985 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:49,986 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:49,985 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1CC56B9B024F4B1DB81D08A7C0782D7F
2016-05-17 22:37:49,986 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:49,986 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:49,986 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:49,986 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:49,986 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:49,987 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:49,987 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:49,987 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:49,987 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:49,987 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1CC56B9B024F4B1DB81D08A7C0782D7F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:49,987 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E5EAE7FADD5C4C2DB7889C3A1C2CAC37, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:49,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:49,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E5EAE7FADD5C4C2DB7889C3A1C2CAC37
2016-05-17 22:37:49,988 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:49,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E5EAE7FADD5C4C2DB7889C3A1C2CAC37
2016-05-17 22:37:49,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E5EAE7FADD5C4C2DB7889C3A1C2CAC37 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:49,989 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:49,989 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:49,989 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:49,989 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524669784_0001_1_00_000000_0
2016-05-17 22:37:49,991 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:49,992 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@67c8ebf
2016-05-17 22:37:49,993 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:49,993 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E5EAE7FADD5C4C2DB7889C3A1C2CAC37
2016-05-17 22:37:49,993 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL"], id: 1CC56B9B024F4B1DB81D08A7C0782D7F
2016-05-17 22:37:49,994 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL"]
2016-05-17 22:37:49,994 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:49,994 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:49,995 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@67c8ebf
2016-05-17 22:37:49,995 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:49,996 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:49,996 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EFC74D2D40B94F58B6ABB49FE5D551AD, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:49,999 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:49,999 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:50,000 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524669784_0001_1_00_000000_0
2016-05-17 22:37:50,000 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524669784_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:50,001 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524669784_0001_1_00_000000_0
2016-05-17 22:37:50,001 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EFC74D2D40B94F58B6ABB49FE5D551AD, mem on close (mb), free: 219, total: 353, max: 672
2016-05-17 22:37:50 Completed running task attempt: attempt_1463524669784_0001_1_00_000000_0
2016-05-17 22:37:50,002 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524669784_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:50,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1308, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370671616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524669996, Process_Duration=5, Process_End_Time=1463524670001, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:50,004 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669784_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EFC74D2D40B94F58B6ABB49FE5D551AD, taskAttemptId=attempt_1463524669784_0001_1_00_000000_0, startTime=1463524669967, finishTime=1463524670003, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1308, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370671616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524669996, Process_Duration=5, Process_End_Time=1463524670001, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:50,004 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:50,004 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524669784_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:50,003 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:50,004 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524669784_0001_1_00_000000_0
2016-05-17 22:37:50,005 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:50,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669784_0001_1][Event:TASK_FINISHED]: vertexName=EFC74D2D40B94F58B6ABB49FE5D551AD, taskId=task_1463524669784_0001_1_00_000000, startTime=1463524669967, finishTime=1463524670004, timeTaken=37, status=SUCCEEDED, successfulAttemptID=attempt_1463524669784_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1308, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370671616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524669996, Process_Duration=5, Process_End_Time=1463524670001, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:50,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:50,005 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:50,005 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524669784_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:50,006 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD] : 1
2016-05-17 22:37:50,006 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD]
2016-05-17 22:37:50,006 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524669784_0001_00_000001
2016-05-17 22:37:50,007 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:50,007 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669784_0001_1][Event:VERTEX_FINISHED]: vertexName=EFC74D2D40B94F58B6ABB49FE5D551AD, vertexId=vertex_1463524669784_0001_1_00, initRequestedTime=1463524669943, initedTime=1463524669954, startRequestedTime=1463524669954, startedTime=1463524669954, finishTime=1463524670006, timeTaken=52, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1308, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370671616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524669996, Process_Duration=5, Process_End_Time=1463524670001, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524669967, firstTasksToStart=[ task_1463524669784_0001_1_00_000000 ], lastTaskFinishTime=1463524670003, lastTasksToFinish=[ task_1463524669784_0001_1_00_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524669784_0001_1_00_000000 ], longestDurationTasks=[ task_1463524669784_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:50,007 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:50,008 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524669784_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:50,008 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669784_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524669784_0001_00_000001, stoppedTime=1463524670008, exitStatus=0
2016-05-17 22:37:50,008 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:50,008 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524669784_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:50,008 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:50,009 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524669784_0001_1
2016-05-17 22:37:50,009 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669784_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524669784_0001_1
2016-05-17 22:37:50,009 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 1CC56B9B024F4B1DB81D08A7C0782D7F for vertex: vertex_1463524669784_0001_1_00 [EFC74D2D40B94F58B6ABB49FE5D551AD]
2016-05-17 22:37:50,017 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524669784_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:50,017 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524669784_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:50,017 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524669784_0001_00_000001, Exiting
2016-05-17 22:37:50,017 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524669784_0001_00_000001
2016-05-17 22:37:50,017 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524669784_0001_00_000001
2016-05-17 22:37:50,019 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524669784_0001_1, queueSize=0
2016-05-17 22:37:50,019 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:50,020 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524669784_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524669784_0001_1, startTime=1463524669942, finishTime=1463524670018, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=1308, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370671616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524669996, Process_Duration=5, Process_End_Time=1463524670001, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:50,020 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524669784_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:50,020 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524669784_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:50,020 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524669784_0001_00_000001
2016-05-17 22:37:50,021 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524669784_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:50 Completed Dag: dag_1463524669784_0001_1
2016-05-17 22:37:50,022 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524669784_0001_00_000001 completed successfully
2016-05-17 22:37:50,021 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524669784_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:50,022 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524669784_0001_1_post
2016-05-17 22:37:50,022 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:50,023 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:50,023 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524669784_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:37:50,024 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524669784_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:37:50,026 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ..._RESULTS.ALL_'field2'_ALL, applicationId=application_1463524669784_0001
2016-05-17 22:37:50,028 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:50,028 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:50,028 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:50,028 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:50,028 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:50,029 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:50,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:50,029 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:50,030 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:50,030 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:50,030 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:50,030 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:50,031 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:50,036 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ACB66502C4A04AFAAA740BA3F8A12758/B6DE6481DD224796BE2D7C04EDB91E01/.tez/application_1463524669784_0001
2016-05-17 22:37:50,037 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_'field2'_ALL/_temporary
2016-05-17 22:37:50,037 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.272
2016-05-17 22:37:50,109 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_'field2'_ALL
2016-05-17 22:37:50,132 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:50,132 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:50,161 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 22:37:50,171 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-17 22:37:50,172 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:50,172 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-05-17 22:37:50,172 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:37:50,185 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 22:37:50,185 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1
2016-05-17 22:37:50,196 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting
2016-05-17 22:37:50,196 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:50,197 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1"]
2016-05-17 22:37:50,197 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] parallel execution of steps is enabled: false
2016-05-17 22:37:50,198 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executing total steps: 1
2016-05-17 22:37:50,198 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] allocating management threads: 1
2016-05-17 22:37:50,199 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] starting step: (1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-17 22:37:50,199 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every.every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:37:50,210 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] tez session mode enabled: true
2016-05-17 22:37:50,214 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:37:50,214 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:37:50,215 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:50,215 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:37:50,216 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:37:50,225 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ADF39EC748EF4CB29E3C3DFDEAE88B5B/BA43266F8EA44CB3B9344CF06CA9690B/.tez/application_1463524670215_0001 doesn't exist and is created
2016-05-17 22:37:50,241 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:37:50,241 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:37:50,241 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ADF39EC748EF4CB29E3C3DFDEAE88B5B/BA43266F8EA44CB3B9344CF06CA9690B/.tez/application_1463524670215_0001
2016-05-17 22:37:50,244 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524670215_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:37:50,256 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:37:50,256 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:37:50,257 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:37:50,258 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:37:50,258 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ADF39EC748EF4CB29E3C3DFDEAE88B5B/BA43266F8EA44CB3B9344CF06CA9690B/.tez/application_1463524670215_0001/localmode-log-dir
2016-05-17 22:37:50,258 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ADF39EC748EF4CB29E3C3DFDEAE88B5B/BA43266F8EA44CB3B9344CF06CA9690B/.tez/application_1463524670215_0001/localmode-log-dir/history.txt.appattempt_1463524670215_0001_000000, maxErrors=10
2016-05-17 22:37:50,258 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:37:50,258 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524670215_0001_000000, appSubmitTime=1463524670243, launchTime=1463524670243
2016-05-17 22:37:50,261 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:37:50,262 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:37:50,263 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:37:50,264 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:37:50,268 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:50816
2016-05-17 22:37:50,269 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:37:50,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524670215_0001_000000, startTime=1463524670271
2016-05-17 22:37:50,271 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:37:50,271 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:37:50,341 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:37:50,341 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:37:50,341 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, applicationId=application_1463524670215_0001, dagName=[ADF39EC748EF4CB29E3C3DFDEAE88B5B/BA43266F8EA44CB3B9344CF06CA9690B] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-17 22:37:50,356 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [ADF39EC748EF4CB29E3C3DFDEAE88B5B/BA43266F8EA44CB3B9344CF06CA9690B] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-17 22:37:50,357 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524670215_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ADF39EC748EF4CB29E3C3DFDEAE88B5B/BA43266F8EA44CB3B9344CF06CA9690B/.tez/application_1463524670215_0001/localmode-log-dir/dag_1463524670215_0001_1.dot
2016-05-17 22:37:50,357 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ADF39EC748EF4CB29E3C3DFDEAE88B5B/BA43266F8EA44CB3B9344CF06CA9690B/.tez/application_1463524670215_0001/localmode-log-dir/dag_1463524670215_0001_1-tez-dag.pb.txt
2016-05-17 22:37:50,361 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524670215_0001_1
2016-05-17 22:37:50,361 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:50,362 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [ADF39EC748EF4CB29E3C3DFDEAE88B5B/BA43266F8EA44CB3B9344CF06CA9690B] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-17 22:37:50 Running Dag: dag_1463524670215_0001_1
2016-05-17 22:37:50,373 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524670215_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524670215_0001_1, submitTime=1463524670356
2016-05-17 22:37:50,373 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:37:50,373 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FA64A67E4216462EA9988FDB71FC606A
2016-05-17 22:37:50,373 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:37:50,374 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524670215_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524670215_0001_1, initTime=1463524670373
2016-05-17 22:37:50,374 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:37:50,374 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524670215_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:37:50,374 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524670215_0001_1][Event:DAG_STARTED]: dagID=dag_1463524670215_0001_1, startTime=1463524670374
2016-05-17 22:37:50,374 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:37:50,374 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, applicationId=application_1463524670215_0001, dagName=[ADF39EC748EF4CB29E3C3DFDEAE88B5B/BA43266F8EA44CB3B9344CF06CA9690B] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1
2016-05-17 22:37:50,375 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:37:50,375 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524670215_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:37:50,375 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FA64A67E4216462EA9988FDB71FC606A : {9D25F32802944F7AB5752CDA6313E93B={InputName=9D25F32802944F7AB5752CDA6313E93B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:37:50,375 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9D25F32802944F7AB5752CDA6313E93B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:37:50,375 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A]
2016-05-17 22:37:50,375 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A]
2016-05-17 22:37:50,375 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A]
2016-05-17 22:37:50,376 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A]. Starting root input initializers: 1
2016-05-17 22:37:50,376 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:37:50,377 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9D25F32802944F7AB5752CDA6313E93B on vertex vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A]
2016-05-17 22:37:50,378 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:37:50,378 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9D25F32802944F7AB5752CDA6313E93B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:37:50,379 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:37:50,379 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:37:50,380 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 22:37:50,381 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9D25F32802944F7AB5752CDA6313E93B on vertex vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A]
2016-05-17 22:37:50,381 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:50,381 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9D25F32802944F7AB5752CDA6313E93B=forAllWorkUnits=true, update=[1]}
2016-05-17 22:37:50,381 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A] parallelism set to 1
2016-05-17 22:37:50,381 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:37:50,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524670215_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524670215_0001_1_00, eventCount=1
2016-05-17 22:37:50,382 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A]
2016-05-17 22:37:50,382 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9989077E8282476CAB56C4E3A09E59C3, vertexId=vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:37:50,382 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9989077E8282476CAB56C4E3A09E59C3, vertexId=vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A]
2016-05-17 22:37:50,383 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9989077E8282476CAB56C4E3A09E59C3, vertexName=FA64A67E4216462EA9988FDB71FC606A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:37:50,384 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9989077E8282476CAB56C4E3A09E59C3, vertexId=vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A]
2016-05-17 22:37:50,385 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:37:50,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524670215_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FA64A67E4216462EA9988FDB71FC606A, vertexId=vertex_1463524670215_0001_1_00, initRequestedTime=1463524670375, initedTime=1463524670385, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:37:50,385 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A] with distanceFromRoot: 0
2016-05-17 22:37:50,385 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:37:50,386 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FA64A67E4216462EA9988FDB71FC606A
2016-05-17 22:37:50,386 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A]
2016-05-17 22:37:50,386 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:37:50,386 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524670215_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524670215_0001_1_00, startRequestedTime=1463524670386, startedTime=1463524670386
2016-05-17 22:37:50,387 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:37:50,387 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:37:50,387 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524670215_0001_1][Event:TASK_STARTED]: vertexName=FA64A67E4216462EA9988FDB71FC606A, taskId=task_1463524670215_0001_1_00_000000, scheduledTime=1463524670387, launchTime=1463524670387
2016-05-17 22:37:50,388 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524670215_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:37:50,388 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A]
2016-05-17 22:37:50,388 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [ADF39EC748EF4CB29E3C3DFDEAE88B5B/BA43266F8EA44CB3B9344CF06CA9690B] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, VertexName: FA64A67E4216462EA9988FDB71FC606A, VertexParallelism: 1, TaskAttemptID:attempt_1463524670215_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9D25F32802944F7AB5752CDA6313E93B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9989077E8282476CAB56C4E3A09E59C3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:50,388 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524670215_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:37:50,389 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:37:50,389 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:37:50,390 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:37:50,390 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:37:50,390 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524670215_0001_00_000001
2016-05-17 22:37:50,390 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524670215_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:37:50,392 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524670215_0001_00_000001
2016-05-17 22:37:50,392 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:37:50,392 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] submitted tez dag to app master: application_1463524670215_0001, with dag id: dag_1463524670215_0001_1
2016-05-17 22:37:50,393 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524670215_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:37:50,393 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:37:50,393 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524670215_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524670215_0001_00_000001, launchTime=1463524670393
2016-05-17 22:37:50,393 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524670215_0001_00_000001
2016-05-17 22:37:50,394 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524670215_0001_1_00_000000_0] to container: [container_1463524670215_0001_00_000001]
2016-05-17 22:37:50,394 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524670215_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:37:50,394 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524670215_0001_1_00_000000_0] started. Is using containerId: [container_1463524670215_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:37:50,395 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524670215_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FA64A67E4216462EA9988FDB71FC606A, taskAttemptId=attempt_1463524670215_0001_1_00_000000_0, startTime=1463524670394, containerId=container_1463524670215_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524670215_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:37:50,395 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524670215_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:37:50,395 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524670215_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:37:50,395 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524670215_0001_00_000001 given task: attempt_1463524670215_0001_1_00_000000_0
2016-05-17 22:37:50,395 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524670215_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524670215_0001_1_00_000000_0
2016-05-17 22:37:50,395 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:37:50 Starting to run new task attempt: attempt_1463524670215_0001_1_00_000000_0
2016-05-17 22:37:50,396 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524670215_0001_1_00_000000_0
2016-05-17 22:37:50,396 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:50,396 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:37:50,396 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:37:50,396 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:37:50,396 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:37:50,396 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:37:50,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [ADF39EC748EF4CB29E3C3DFDEAE88B5B/BA43266F8EA44CB3B9344CF06CA9690B] every.every/(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, VertexName: FA64A67E4216462EA9988FDB71FC606A, VertexParallelism: 1, TaskAttemptID:attempt_1463524670215_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9D25F32802944F7AB5752CDA6313E93B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9989077E8282476CAB56C4E3A09E59C3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:37:50,397 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:37:50,397 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524670215_0001_1_00_000000_0
2016-05-17 22:37:50,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:37:50,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:50,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:50,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9D25F32802944F7AB5752CDA6313E93B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:37:50,399 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524670215_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:37:50,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:37:50,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9D25F32802944F7AB5752CDA6313E93B
2016-05-17 22:37:50,400 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9D25F32802944F7AB5752CDA6313E93B
2016-05-17 22:37:50,401 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:37:50,402 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:37:50,402 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9D25F32802944F7AB5752CDA6313E93B
2016-05-17 22:37:50,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9D25F32802944F7AB5752CDA6313E93B
2016-05-17 22:37:50,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9989077E8282476CAB56C4E3A09E59C3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:37:50,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:37:50,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9989077E8282476CAB56C4E3A09E59C3
2016-05-17 22:37:50,404 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:37:50,404 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9989077E8282476CAB56C4E3A09E59C3
2016-05-17 22:37:50,409 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FA64A67E4216462EA9988FDB71FC606A, ordinal: 0
2016-05-17 22:37:50,409 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA64A67E4216462EA9988FDB71FC606A, mem on start (mb), free: 143, total: 353, max: 672
2016-05-17 22:37:50,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:37:50,411 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:37:50,411 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:37:50,415 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:37:50,415 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9989077E8282476CAB56C4E3A09E59C3
2016-05-17 22:37:50,415 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:37:50,416 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:37:50,416 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:37:50,416 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:50,416 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:37:50,417 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:37:50,417 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:37:50,417 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-05-17 22:37:50,417 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9D25F32802944F7AB5752CDA6313E93B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:37:50,418 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9989077E8282476CAB56C4E3A09E59C3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:37:50,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:37:50,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9D25F32802944F7AB5752CDA6313E93B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:37:50,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9D25F32802944F7AB5752CDA6313E93B
2016-05-17 22:37:50,419 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:50,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9D25F32802944F7AB5752CDA6313E93B
2016-05-17 22:37:50,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:37:50,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:37:50,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:37:50,420 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524670215_0001_1_00_000000_0
2016-05-17 22:37:50,420 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:37:50,421 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2da6066
2016-05-17 22:37:50,421 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2da6066
2016-05-17 22:37:50,422 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:37:50,422 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:37:50,423 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:50,424 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 9D25F32802944F7AB5752CDA6313E93B
2016-05-17 22:37:50,424 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1"], id: 9989077E8282476CAB56C4E3A09E59C3
2016-05-17 22:37:50,424 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1"]
2016-05-17 22:37:50,424 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 22:37:50,424 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:37:50,425 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FA64A67E4216462EA9988FDB71FC606A, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:37:50,427 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:37:50,427 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:37:50,427 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524670215_0001_1_00_000000_0
2016-05-17 22:37:50,427 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524670215_0001_1_00_000000_0 given a go for committing the task output.
2016-05-17 22:37:50,428 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524670215_0001_1_00_000000_0
2016-05-17 22:37:50,428 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA64A67E4216462EA9988FDB71FC606A, mem on close (mb), free: 141, total: 353, max: 672
2016-05-17 22:37:50 Completed running task attempt: attempt_1463524670215_0001_1_00_000000_0
2016-05-17 22:37:50,429 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524670215_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:37:50,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524670215_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FA64A67E4216462EA9988FDB71FC606A, taskAttemptId=attempt_1463524670215_0001_1_00_000000_0, startTime=1463524670394, finishTime=1463524670430, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370671616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524670425, Process_Duration=3, Process_End_Time=1463524670428, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:50,430 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524670215_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:37:50,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:37:50,430 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:37:50,430 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524670215_0001_1_00_000000_0
2016-05-17 22:37:50,431 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370671616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20][cascading.flow.SliceCounters Process_Begin_Time=1463524670425, Process_Duration=3, Process_End_Time=1463524670428, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20, Tuples_Written=20]]
2016-05-17 22:37:50,431 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:37:50,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524670215_0001_1][Event:TASK_FINISHED]: vertexName=FA64A67E4216462EA9988FDB71FC606A, taskId=task_1463524670215_0001_1_00_000000, startTime=1463524670394, finishTime=1463524670431, timeTaken=37, status=SUCCEEDED, successfulAttemptID=attempt_1463524670215_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370671616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524670425, Process_Duration=3, Process_End_Time=1463524670428, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:50,431 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524670215_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:37:50,432 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A] : 1
2016-05-17 22:37:50,432 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A]
2016-05-17 22:37:50,432 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:37:50,432 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:50,432 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524670215_0001_00_000001
2016-05-17 22:37:50,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524670215_0001_1][Event:VERTEX_FINISHED]: vertexName=FA64A67E4216462EA9988FDB71FC606A, vertexId=vertex_1463524670215_0001_1_00, initRequestedTime=1463524670375, initedTime=1463524670385, startRequestedTime=1463524670386, startedTime=1463524670386, finishTime=1463524670432, timeTaken=46, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370671616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524670425, Process_Duration=3, Process_End_Time=1463524670428, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20, vertexStats=firstTaskStartTime=1463524670394, firstTasksToStart=[ task_1463524670215_0001_1_00_000000 ], lastTaskFinishTime=1463524670430, lastTasksToFinish=[ task_1463524670215_0001_1_00_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524670215_0001_1_00_000000 ], longestDurationTasks=[ task_1463524670215_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:37:50,433 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:37:50,432 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:37:50,433 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524670215_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:37:50,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524670215_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524670215_0001_00_000001, stoppedTime=1463524670433, exitStatus=0
2016-05-17 22:37:50,434 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524670215_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:37:50,434 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-05-17 22:37:50,434 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524670215_0001_1
2016-05-17 22:37:50,434 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:37:50,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524670215_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524670215_0001_1
2016-05-17 22:37:50,436 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 9989077E8282476CAB56C4E3A09E59C3 for vertex: vertex_1463524670215_0001_1_00 [FA64A67E4216462EA9988FDB71FC606A]
2016-05-17 22:37:50,443 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524670215_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:37:50,443 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524670215_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:37:50,443 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524670215_0001_00_000001, Exiting
2016-05-17 22:37:50,444 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524670215_0001_00_000001
2016-05-17 22:37:50,444 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524670215_0001_00_000001
2016-05-17 22:37:50,446 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524670215_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:37:50,446 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524670215_0001_00_000001 completed successfully
2016-05-17 22:37:50,447 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524670215_0001_1, queueSize=0
2016-05-17 22:37:50,448 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:37:50,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524670215_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524670215_0001_1, startTime=1463524670374, finishTime=1463524670446, timeTaken=72, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=77, FILE_BYTES_WRITTEN=152, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370671616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, cascading.flow.SliceCounters, Process_Begin_Time=1463524670425, Process_Duration=3, Process_End_Time=1463524670428, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 22:37:50,449 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524670215_0001_1 finished with state: SUCCEEDED
2016-05-17 22:37:50,449 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524670215_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:37:50,449 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524670215_0001_00_000001
2016-05-17 22:37:50,450 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524670215_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:37:50 Completed Dag: dag_1463524670215_0001_1
2016-05-17 22:37:50,450 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524670215_0001_1, dagState=SUCCEEDED
2016-05-17 22:37:50,451 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524670215_0001_1_post
2016-05-17 22:37:50,451 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:37:50,451 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:37:50,455 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...d'_RESULTS.ALL_UNKNOWN_-1, applicationId=application_1463524670215_0001
2016-05-17 22:37:50,457 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:37:50,458 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:37:50,458 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:37:50,458 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:37:50,458 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:37:50,458 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:37:50,458 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:37:50,459 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:37:50,459 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:37:50,459 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:37:50,459 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:37:50,459 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:37:50,459 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:37:50,464 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/ADF39EC748EF4CB29E3C3DFDEAE88B5B/BA43266F8EA44CB3B9344CF06CA9690B/.tez/application_1463524670215_0001
2016-05-17 22:37:50,464 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/everyeverypipeassemblyplatform/-1_'field'_RESULTS.ALL_UNKNOWN_-1/_temporary
2016-05-17 22:37:50,464 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] completed in: 00:00.268
2016-05-17 22:37:50,530 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_UNKNOWN_-1
2016-05-17 22:37:50,552 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:37:50,552 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:37:50,593 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 22:37:50,597 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-17 22:37:50,597 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:50,598 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every.every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 22:37:50,598 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/everyeverypipeassemblyplatform//-1_'field'_RESULTS.ALL_UNKNOWN_'field2'